EMPIRICAL  ANALYSIS  OF  VARIOUS 
MULTI-DIMENSIONAL  KNAPSACK  HEURISTICS 

THESIS 


Yong  Kun  Cho,  Captain,  Republic  of  Korea  Army 


AFIT/GOR/ENS/02-04 


DEPARTMENT  OF  THE  AIR  FORCE 
AIR  UNIVERSITY 

AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 


Wright-Patterson  Air  Force  Base,  Ohio 


APPROVED  EOR  PUBEIC  REEEASE;  DISTRIBUTION  UNLIMITED. 


Report  Documentation  Page 

Report  Date 

Report  Type 

Dates  Covered  (from...  to) 

8  Mar  2002 

Final 

Jun  2001  -  Mar  2002 

Title  and  Subtitle 

Contract  Number 

Empirical  Analysis  of  Various  Multi-Dimensional 

Knapsack  Heuristics 

Grant  Number 

Program  Element  Number 


Author(s)  Project  Number 

Capt  Yong  Kun  Cho,  Repulic  of  Korea  Army 

Task  Number 
Work  Unit  Number 

Performing  Organization  Name(s)  and 
Address(es) 

Air  Force  Institute  of  Technology  Graduate  School  of 
Engineering  and  Management  (AFIT/EN)  2950  P 
Street,  Bldg  640  WPAFB,  OH  45433-7765 


Sponsoring/Monitoring  Agency  Name(s)  and 

Sponsor/Monitor’s  Acronym(s) 

Address(es) 

Sponsor/Monitor’s  Report  Number(s) 

Distribution/Availability  Statement 

Approved  for  public  release,  distribution  unlimited 

Supplementary  Notes 

Abstract 

Since  the  multidimensional  knapsack  problems  are  NP-hard  problems,  the  exact  solutions  of  knapsack 
problems  often  need  excessive  computing  time  and  storage  space.  Thus,  heuristic  approaches  are  more 
practical  for  multidimensional  knapsack  problems  as  problems  get  large.  This  thesis  presents  the  results  of 
an  empirical  study  of  the  performance  of  heuristic  solution  procedures  based  on  the  coefficients 
correlation  structures  and  constraint  slackness  settings.  In  this  thesis,  the  three  representative  greedy 
heuristics,  Toyoda,  Senju  and  Toyoda,  and  Loulou  and  Michaelides  methods,  are  studied.  The  purpose  of 
this  thesis  is  to  explore  which  heuristic  of  the  three  representative  greedy  heuristics  perform  best  under 
certain  combination  of  conditions  between  constraint  slackness  and  correlation  structures.  This  thesis 
examines  three  heuristics  over  1 120  problems  which  are  all  2KPs  with  100  variables  created  by  four 
constraint  slackness  settings  and  45  feasible  correlation  structures.  Then  we  analyze  why  the  best  heuristic 
behaves  as  it  does  as  a  function  of  problem  characteristic.  The  last  chapter  presents  two  new  heuristics 
using  knowledge  gained  in  the  study.  When  these  new  heuristics  are  competitively  tested  against  the  three 
original  heuristics,  the  results  show  their  better  performances. 

Subject  Terms 

Multi-Dimensional  Knapsack  Problem,  Heuristic,  Computational  Testing 


Performing  Organization  Report  Number 

AFIT/GOR/ENS/02-04 


Report  Classification 

Classification  of  this  page 

unclassified 

unclassified 

Classification  of  Abstract 

Limitation  of  Abstract 

unclassified 

UU 

Number  of  Pages 

92 

The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  refleet  the  official 
policy  or  position  of  the  United  States  Air  Force,  Department  of  Defense,  or  the  U.  S. 
Government. 


AFIT/GOR/ENS/02-04 


EMPIRICAL  ANALYSIS  OE  VARIOUS 
MULTI-DIMENSIONAL  KNAPSACK  HEURISTICS 

THESIS 


Presented  to  the  Eaeulty 
Department  of  Operational  Seienees 
Graduate  Sehool  of  Engineering  and  Management 
Air  Eorce  Institute  of  Technology 
Air  University 

Air  Education  and  Training  Command 
In  Partial  Eulfillment  of  the  Requirements  for  the 
Degree  of  Master  of  Science  in  Operations  Research 


Yong  Kun  Cho,  BS 
Captain,  Republic  of  Korea  Army 

March  2002 


APPROVED  EOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 


AFIT/GOR/ENS/02-04 


EMPIRICAL  ANALYSIS  OE  VARIOUS 
MULTI-DIMENSIONAL  KNAPSACK  HEURISTICS 


Yong  Kun  Cho,  B.S. 
Captain,  Republic  of  Korea  Army 


Approved; 


Raymond  R.  Hill,  Lt  Col,  USAE  (Advisor) 


date 


James  T.  Moore,  Ph.D.  (Reader) 


date 


Acknowledgments 


First  and  foremost,  I’d  like  to  express  my  deepest  gratitude  to  my  thesis  advisor, 
Lt.  Col  Raymond  R.  Hill,  for  taking  the  ehallenge  of  guiding  me  through  this  thesis 
effort.  Without  his  guidance  and  endless  support,  I  would  not  have  dreamed  of  coming  to 
such  a  state  of  accomplishment.  The  experience,  insight  and  knowledge  he  has  shared 
with  me  are  invaluable  and  well  appreciated.  I  would  also  like  to  thank  Dr.  James  T. 
Moore  for  his  interest  and  support. 

In  addition,  I  am  indebted  to  the  Korean  Army  for  providing  me  with  this 
wonderful  opportunity  of  Master’s  degree  program  that  has  broadened  my  knowledge  as 
well  as  stimulated  my  understanding  of  military  technology.  Hopefully,  all  this  will  help 
me  better  serve  my  country  in  the  future. 

Most  importantly.  I’m  forever  thankful  to  my  wife  and  son  for  being  there  with 
love,  support  and  understanding.  I  also  feel  grateful  for  my  parents  and  in-law’s  in  Korea 
for  their  support  and  encouragement.  Thank  you  everyone  for  your  patience,  endurance, 
and  strong  belief  in  me. 


Yong  Kun  Cho 


Table  of  Contents 


Page 

Acknowledgments . iv 

List  of  Figures . vii 

List  of  Tables . viii 

Abstract . ix 

L  Introduction . 1 

1.1  Motivation . 1 

1.2  Background . 2 

1.2.1  The  0-1  Knapsack  Problem . 2 

1.2.2  The  multidimensional  knapsaek  problem  (MKP) . 2 

1.3  Problem . 3 

1.4  Objeetive . 3 

1.5  Overview . 4 

11.  Review  of  Related  Literature . 5 

2.1  Introduetion . 5 

2.2  An  Empirical  Science . 5 

2.3  Empirieal  analysis  ofheuristies . 6 

2.3.1  Knapsaek  Problem . 6 

2.3.2  Greedy  algorithms  for  multi-dimensional  knapsaek  problems . 7 

2.3.3  Experimental  eomparison  ofheuristies  for  the  knapsaek  problem . 12 

2.3.4  Studies  into  effeets  of  correlation  on  solution  proeedure  performance....  14 

ITT.  Methodology . 18 

3.1  Introduetion . 18 

3.2  Test  problem  characteristics  in  the  library . 18 

3.3  Approaehes  to  the  empirical  analysis  of  algorithms  and  heuristics . 20 

3.4  Computer  coding  of  three  heuristie  methods . 21 

3.5  Analysis  of  Results . 24 

3.5.1  Relative  Error . 24 

3.5.2  Chi-square  test . 25 

3.5.3  Sign  test  for  paired  samples . 26 

3.6  New  heuristic . 26 

3.6.1  New  Combined  Heuristie . 26 

3.6.2  Cho  Heuristie . 27 


V 


Page 

IV,  Analysis  of  Legacy  Heuristics . 28 

4. 1  Constraints  slackness . 28 

4.1.1  Results . 28 

4.1.2  Analysis  of  constraint  slackness  of  Si  =  0.3  and  S2  =  0.3  (1,1) . 30 

4.1.3  Analysis  of  mixed  slackness,  Si  S2,  (1,2)  or  (2,1) . 31 

4.1.4  Analysis  of  constraint  slackness  of  Si  =  0.7  and  S2  =  0.7  (2,2) . 35 

4.1.5  Implication . 36 

4.2  Correlation . 36 

4.2.1  Results . 37 

4.2.2  Chi  -  square  test . 41 

4.2.3  Analysis  of  =-0.99997,  =- 0.99773,  and  p.^^  =0.99752 

(-2,  -2,  2) . 42 

4.2.4  Analysis  of  Toyoda’s  heuristic . 43 

4.2.5  Analysis  of  Senju-Toyoda’s  heuristic . 44 

4.2.6  Analysis  of  Loulou-  Michaelides’  heuristic . 45 

4.2.7  Implication . 46 

V,  New  Heuristic  Comparison . 47 

5.1  Introduction . 47 

5.2  New  Combined  Heuristic . 47 

5.2.1  Results  under  Constraint  Slackness . 49 

5.2.2  Results  under  Correlation  Structure . 50 

5.3  Cho  Heuristic . 54 

5.3.1  General  Results . 56 

5.3.2  Results  under  various  constraint  slackness . 56 

VL  Conclusion . 58 

6.1  Summary . 58 

6.2  Findings . 58 

6.3  Recommendations . 59 

Appendix  A,  TOYODA  Heuristic  Code . 61 

Appendix  B,  S  -  T  Heuristic  Code . 64 

Appendix  C.  L  -  M  Heuristic  Code . 67 

Appendix  D,  New  Combined  Heuristic  Code . 70 

Appendix  E,  Cho  Heuristic  Code . 75 

Bibliography . 79 

Vita . 81 


VI 


List  of  Figures 


Page 

Figure  1.  Counts  of  best  of  three  heuristic  methods  under  constraint  slackness . 28 

Figure  2.  Effective  length  of  withdrawal . 32 

Figure  3.  Performance  of  3  heuristic  methods  in  terms  of  best  constraint  resource  use. ..33 
Figure  4.  Performance  of  3  heuristic  methods  in  terms  of  best  constraint  resource  use.  ..34 
Figure  5.  Performance  of  the  three  heuristic  methods  when  =2(1) . 38 

Figure  6.  Performance  of  the  three  heuristic  methods  when  =  1  (2) . 39 

Figure  7.  Performance  of  the  three  heuristic  methods  when  =  0  (3) . 39 

Figure  8.  Performance  of  the  three  heuristic  methods  when  p^^^  =  -1  (4) . 40 

Figure  9.  Performance  of  the  three  heuristic  methods  when  p^^^  =  -2  (5) . 40 

Figure  10.  Better  Performance  of  TOYODA  method  under  various  correlations . 43 

Figure  11.  Better  Performance  of  S  -  T  method  under  various  p^x^i  correlations  (1)  ....44 

Figure  12.  Better  Performance  of  S  -  T  method  under  various  correlations  (2) . 44 

Figure  13.  Better  Performance  of  L  -  M  method  under  various  correlations . 45 

Figure  14.  Performance  of  new  combined  method  under  various  constraint  slackness  ...50 
Figure  15.  Performance  of  new  combined  heuristic  when  =  2  (1) . 51 

Figure  16.  Performance  of  new  combined  heuristic  when  =  1  (2) . 51 

Figure  17.  Performance  of  new  combined  heuristic  when  =  0  (3) . 52 

Figure  18.  Performance  of  new  combined  heuristic  when  p^^,  =  -1  (4) . 52 

Figure  19.  Performance  of  new  combined  heuristic  when  =  -2  (5) . 52 

Figure  20.  Performance  of  Cho  heuristic  under  various  constraint  slackness . 57 

vii 


List  of  Tables 


Page 

Table  1 .  Results  by  Three  Heuristics  Solutions  over  1120  problems . 23 

Table  2.  Relative  Error  of  Three  Heuristics . 24 

Table  3.  The  number  of  times  best  by  each  heuristic  under  constraint  slackness . 29 

Table  4.  Chi-square  test  for  constraint  slackness . 29 

Table  5.  Sign  test,  TOYODA  vs.  L  -  M  under  Si  =  0.3  and  S2  =  0.3 . 30 

Table  6.  The  number  of  times  least  slack  remained  in  each  constraint  (Excluding  ties). .34 

Table  7.  The  number  of  times  least  slack  remained  in  each  constraint  (Excluding  ties)..  3  5 

Table  8.  Sign  test,  TOYODA  vs.  E  -  M  under  Si  =  0.7  and  S2  =  0.7 . 35 

Table  9.  The  number  of  times  best  by  each  heuristic  under  correlation . 37 

Table  10.  Chi-square  test  for  correlations . 41 

Table  11.  Number  of  times  optimum  found  by  each  heuristic  under  correlation  structure 
-2,  -2,2 . 42 

Table  12.  Dominant  Constraint  Slackness  and  the  Best  Heuristic . 48 

Table  13.  Dominant  Correlation  Structures  and  the  Best  Heuristic . 48 

Table  14.  Dominant  Combination  between  Constraint  Slackness  and  Correlation 

Structure,  and  the  Best  Heuristic . 49 

Table  15.  The  number  of  times  best  by  each  heuristic  under  constraint  slackness . 50 

Table  16.  The  number  of  times  best  by  the  new  combined  heuristics  under  correlation 

structure . 53 

Table  17.  Comparison  of  CHO  heuristic  with  test  heuristics . 56 

Table  18.  The  number  of  times  best  by  each  heuristic  under  constraint  slackness . 57 


viii 


AFIT/GOR/ENS/02-04 


Abstract 

Since  the  multidimensional  knapsack  problems  are  NP-hard  problems,  the  exact 
solutions  of  knapsack  problems  often  need  exeessive  computing  time  and  storage  space. 
Thus,  heuristic  approaches  are  more  praetical  for  multidimensional  knapsaek  problems  as 
problems  get  large.  This  thesis  presents  the  results  of  an  empirical  study  of  the 
performance  of  heuristic  solution  procedures  based  on  the  eoefficients  correlation 
structures  and  constraint  slackness  settings.  In  this  thesis,  the  three  representative  greedy 
heuristics,  Toyoda,  Senju  and  Toyoda,  and  Loulou  and  Michaelides’  methods,  are 
studied.  The  purpose  of  this  research  is  to  explore  which  heuristic  of  the  three 
representative  greedy  heuristics  performs  best  under  certain  combinations  of  conditions 
between  constraint  slackness  and  correlation  structures.  This  thesis  examines  three 
heuristics  over  1120  problems  which  are  all  the  two-dimensional  knapsaek  problems 
(2KPs)  with  100  variables  ereated  by  four  eonstraint  slaekness  settings  and  45  feasible 
correlation  struetures.  Then  we  analyze  why  the  best  heuristic  behaves  as  it  does  as  a 
funetion  of  problem  eharaeteristies.  Finally  we  present  two  new  heuristics  using 
knowledge  gained  in  the  study.  When  these  new  heuristies  are  competitively  tested 
against  the  three  representative  greedy  heuristies,  the  results  show  the  new  heuristics 
perform  better. 
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EMPIRICAL  ANALYSIS  OL  VARIOUS 


MULTI-DIMENSIONAL  KNAPSACK  HEURISTICS 

I.  Introduction 


1.1  Motivation 

Obtaining  an  exact  solution  to  an  integer  programming  problem  in  real  practice  is 
sometimes  less  practical  in  comparison  to  an  easily  computed  method  of  acquiring  near- 
optimal  solutions  via  heuristics.  As  problems  get  large,  exact  solutions  often  need 
excessive  computing  time  and  storage  space.  Many  times,  these  large  problems  are 
merely  estimates  of  reality  so  an  optimal  solution  does  not  have  much  meaning. 
Considering  the  imprecision  of  real-world  problem  data,  and  that  a  precise  solution  in 
reality  may  be  meaningless,  obtaining  a  near-optimal  solution  in  a  reasonable  running 
time  may  better  satisfy  a  practitioner  in  the  real  world. 

The  knapsack  problem  has  wide  application  in  many  areas.  Its  general  form,  the 
multidimensional  knapsack  problem  (MKP),  has  frequently  been  used  to  model  various 
decision-making  processes  such  as  resource-allocation,  cargo  loading,  capital  budgeting 
and  cutting  stock  problems.  As  with  other  combinatorial  problems,  computation  time 
increases  rapidly  with  problem  size.  Many  authors  have  developed  heuristic  methods  to 
solve  MKPs.  Greedy  algorithms  are  commonly  used  and  often  yield  good  solutions. 
These  approaches  vary  in  how  they  treat  the  problem  and  select  items  for  inclusion  in  the 
knapsack.  Little,  if  anything,  has  been  done  to  understand  how  heuristic  differences 
affect  performance.  This  thesis  provides  empirical  analyses  of  heuristics  for 
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multidimensional  knapsack  problems  and  examines  how  various  heuristics  function 
according  to  particular  test  problem  characteristics. 

1.2  Background 

1.2.1  The  0-1  Knapsack  Problem 

Suppose  there  are  n  projects.  The yth project, y  =  has  a  cost  aj  and  a  value 

Cj.  A  project  is  either  picked  or  rejected.  There  is  a  resource  limit  of  b  available  for 
which  the  projects  compete.  The  problem  of  choosing  a  subset  of  projects  to  maximize 
the  sum  of  the  values  while  not  exceeding  the  resource  constraint  is  the  0-1  knapsack 
problem, 

{n  n 

•  (1) 

This  problem  is  called  the  knapsack  problem  because  of  the  analogy  to  a  hiker’s 
problem  of  deciding  what  should  be  put  in  a  knapsack  given  a  weight  or  volume 
limitation  on  how  much  can  be  carried.  In  general,  problems  of  this  sort  may  have  m 
constraints.  We  then  refer  to  the  problem  as  the  multidimensional  knapsack  problem 
(MKP)  (Nemhauser  and  Wolsey,  1988).  With  the  MKP,  project  selection  must 
simultaneously  satisfy  all  m  constraints. 

1.2.2  The  multidimensional  knapsack  problem  (MKP) 

The  MKP  is  a  0-1  programming  problem  of  the  following  form; 

Maximize 

(2) 

y=i 
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subject  to 


Y,ayXj<b.  i  =  \,2,...,m  (3) 

y=i 

Xy=0  or  \  7  =  1, 2,  .  .  .  ,  n  (4) 

where  >  0  and  all  Uy  >  0  .  Additionally  at  least  one  >  0  for  each  j.  A  special  case 
of  the  MKP  is  the  two-dimensional  knapsack  problem  (2KP),  where  m  =  2. 

1.3  Problem 

Many  authors  have  developed  approximate  heuristic  methods  for  MKPs  (Senju 
and  Toyoda,  1968;  Toyoda,  1975;  Loulou  and  Michaelides,  1979;  Pirkul,  1987).  Most 
competitively  test  their  heuristic  against  other  heuristics  or  against  some  common 
problem  set.  Few  ever  focus  on  why  their  approach  does  well  on  some  problems  but  not 
so  well  on  other  problems.  In  short,  little  has  been  done  to  determine  why  a  heuristic 
does  well. 

1.4  Objective 

The  objective  of  this  thesis  is  to  understand  what  makes  a  “best”  heuristic.  In 
other  words,  why  does  the  best  heuristic  work  well  and  why  do  the  poorer  heuristics  not 
work  so  well.  This  is  difficult  to  answer  since  performance  varies  by  type  of  problem. 

We  will  focus  on  constraint  tightness,  correlation  between  objective  function  and 
constraints,  and  interconstraint  correlation.  This  is  done  through  the  generalizing  of  test 
problems,  coding  heuristic  methods  (Senju  and  Toyoda,  Toyoda,  and  Loulou  and 
Michaelides  method),  running  all  computer  algorithms  against  a  common  test  set,  and 
analyzing  the  results.  The  comparison  of  these  three  different  methods  on  generalized 
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test  problems  is  based  on  a  relative  error  measure.  The  relative  error  is  defined  as 


{Zqpj.  -  Zqpp  where  Zheu  is  the  heuristie  objeetive  funetion  value  and  Zqpt  is  the 
optimal,  or  best  known  objeetive  funetion  value  for  the  problem.  The  minimum  relative 
error  gives  a  better  solution  as  that  solution  is  eloser  to  the  optimal  solution.  Raw 
objeetive  funetion  values  are  also  used  when  eompiling  eounts  of  best  performing 
heuristie. 

1,5  Overview 

Chapter  2  provides  relevant  baekground  information.  It  traees  an  overview  of 
empirieal  analyses  of  heuristies  and  those  eoneepts  that  apply  to  the  empirieal  analysis  of 
algorithms.  Chapter  3  then  deseribes  how  to  examine  the  three  representative  greedy 
heuristies,  how  to  analyze  eomparative  data  and  how  to  eompare  heuristie  performanee. 
Chapter  4  presents  the  results  of  an  empirieal  study  of  the  three  representative  greedy 
heuristies.  Next,  Chapter  5  presents  two  new  heuristies  and  their  results.  Finally, 

Chapter  6  summarizes  this  work's  findings,  outlines  eontributions  of  this  work,  and 
identifies  areas  for  further  researeh. 
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II. 


Review  of  Related  Literature 


2.1  Introduction 

The  objective  of  this  literature  review  is  to  provide  an  overview  of  empirical 
analyses  of  heuristics  and  those  concepts  that  apply  to  the  empirical  analysis  of 
algorithms.  Additionally,  heuristics  for  multi-dimensional  knapsack  problems  are 
discussed. 

2.2  An  Empirical  Science 

Performance  of  algorithms  may  be  analyzed  in  two  ways.  One  is  to  analyze 
performance  analytically  relying  on  the  methods  of  deductive  mathematics.  The  other  is 
to  analyze  performance  empirically  using  computational  experiments.  The  former, 
mathematical,  methods  are  further  developed  than  the  empirical  science.  However,  the 
mathematic  results  do  not  usually  indicate  how  an  algorithm  is  going  to  perform  on  real 
problems.  If  we  want  to  know  how  an  algorithm  works  on  typical  problems, 
computational  experiments  give  much  more  insight  into  heuristic  performance. 

Hooker  (1994)  suggested,  “Empirical  science  involves  theory.”  Through 
empirical  analysis,  we  can  determine  how  algorithms  work  and  why  algorithms  perform 
well  or  poorly.  Thus,  we  should  analyze  algorithms  to  gain  insight  into  theory. 

Hooker  said,  “  An  empirical  science  of  algorithms  would  immediately  sidestep 
several  of  the  problems  that  beset  a  purely  deductive  science.”  Hooker  supports  this  by 
stating  that  an  empirical  science 

•  does  not  rely  on  proving  hard  worst-case  and  average-case  theorems; 

•  unlike  worst-case  analysis,  can  focus  on  typical  problems; 


5 


•  unlike  average-case  analysis,  need  not  restrict  itself  to  a  simple  and  unrealistic 
distribution  of  random  problems; 

•  can  finesse  the  issue  of  characterizing  a  typical  class  of  problems 
(Hooker,  1994). 

2,3  Empirical  analysis  of  heuristics 

An  efficient  heuristic  provides  a  correct  solution,  not  necessarily  exact  or  optimal, 
in  a  reasonable  amount  of  time  and  resources.  Heuristics  are  generally  efficient  in  terms 
of  solution  quality,  computer  resource  requirements  and  computer  solving  time. 

Complexity  theory  classifies  problems  according  to  their  solution  difficulty. 

Many  problems  are  easy  to  solve  as  there  exists  a  provably  polynomial  time  algorithm  for 
the  problem  where  polynomial  means  algorithm  run  time  is  some  polynomial  function  of 
problem  size.  Many  other  problems  are  difficult  to  solve  as  there  is  no  provable 
polynomial  time  algorithm  for  the  problems.  These  are  referred  to  as  NP  problems. 

NP  Problems  are  very  difficult  to  solve  and  sometimes,  impossible  to  solve  in 
reasonable  time.  Heuristics,  however,  offer  a  viable  alternative.  With  computing  power 
increasing,  heuristics  and  in  particular  local  search,  have  come  into  their  own  in  the 
optimization  world.  The  next  section  introduces  knapsack  and  greedy  MKP  heuristics. 
We  leave  local  search  heuristics  for  the  MKP  as  a  future  research  area. 

2,3.1  Knapsack  Problem. 

The  objective  of  the  knapsack  problem  is  to  maximize  the  value  of  the  items 
selected  without  exceeding  the  resource  limits. 

{n  n 
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To  solve  a  knapsack  problem  using  a  heuristic  algorithm,  letXy  be  a  project  of 
value  Cj,  and  resource  cost  a,.  The  importance  of  the  project  can  be  determined  by 
considering  the  ratio  Cj/uj .  Sequence  the  projects  so  the  following  holds, 

^  ^  >  .  .  .  >  ^  (2) 

fi 

and,  considering  each  item  in  non-increasing  value -resource  ratio,  add  as  many  items  as 
possible,  into  the  knapsack  until  it  is  full. 

2,3.2  Greedy  algorithms  for  multi-dimensional  knapsack  problems 

The  MKP  is  encountered  when  one  has  to  decide  how  to  choose  projects  to  satisfy 
multiple  resource  constraints.  Many  effective  solution  procedures  for  the  MKP  have 
been  developed.  Each  procedure  uses  some  penalty  cost  (vy)  to  quantify  the  relative 
worth  of  the  project. 

Toyoda  (1975)  defined  an  effective  gradient  as  a  means  to  find  a  good 
approximate  solution  to  the  MKP.  Toyoda’s  effective  gradient  is  a  measure  of  element 
worth  per  unit  cost  in  terms  of  all  resources  used.  Toyoda  first  normalized  each 
constraint,  so  all  right-hand  side  (RHS)  values  were  1 .  His  penalty  cost  is  defined  as 
follows: 


«i  an 

ZU-j  u- 

!=1  IV~'  yj  fl\2 


(3) 


where  is  resource  used  in  constraint  i,  and  0  <  <  1  for  /  =  1,. . .,  m  where  m  is  the 

number  of  constraints.  Toyoda’s  method  starts  with  an  empty  knapsack  and  then  adds 
the  element  with  the  highest  scoring  effective  gradient  until  a  knapsack  constraint  is  met. 
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The  value  Vj  is  updated  at  each  iteration.  It  is  called  a  primal  effective  gradient  method 
because  the  solution  is  always  feasible.  An  overview  of  this  approach  is  as  follows: 


Step  1 :  Start  with  all  items  designated  as  not  contained  in  the  knapsacks. 

Step  2:  Compute  the  effective  gradient  for  each  candidate  element  not  currently 
in  the  knapsack  and  feasible:  G.  =  V  . 

Step  3:  Order  the  elements  in  descending  order  according  to  their  effective 
gradient  measures. 

Step  4:  Add  highest  scoring  element  to  knapsacks,  retaining  problem  feasibility. 
Senju  and  Toyoda’s  (1968)  method  is  very  similar  to  Toyoda’s  method. 

However,  their  approach  starts  with  all  items  designated  as  contained  in  the  knapsack. 

The  heuristic  then  drops  projects  {xj)  according  to  ascending  order  of  a  dual  effective 
gradient  until  feasibility  is  achieved.  Their  approach  is  a  two-pass  algorithm  since  the 
authors  realized  the  computed  gradients  might  cause  the  algorithm  to  “over  shoot”  the 
feasibility  target.  Thus,  once  feasible,  the  Senju  -  Toyoda  method  will  attempt  to  restore 
some  items  previously  dropped.  To  define  their  penalty  cost  (vy),  let  ay  be  constraint 
coefficient  of y'th  constraint.  Then,  Pi  =  {an,  aa,  . . .,  a™}  is  the  vector  of  resource  costs  for 

m 

element  i  of  n  elements.  Let  R=  { i  =  1,  ...,n}  be  the  vector  of  total  resource 

j=i 

consumed  in  each  of  m  resource  constraints,  and  L  =  {bi,  b2,  . . .,  bm}  be  the  vector  of 
right-  hand  side  coefficients  and  finally  slack  variables:  S  =  R-  L. 


Consequently,  a  dual  effective  gradient  is:  G,  =  c, 


'  1*^1  ^ 


.  An  overview  of  this 


dual  approach  is  as  follows: 

Step  1 :  Start  with  all  items  designated  as  contained  in  the  knapsacks. 


Step  2:  Compute  an  effective  gradient  for  each  element. 

Step  3:  Order  the  elements  in  ascending  order  according  to  their  effective 
gradient  measures. 

Step  4:  Drop  elements  from  lowest  effective  gradient  measure  until  feasibility 
with  respect  to  all  constraints  is  achieved. 

Step  5:  Re-consider  dropped  elements  for  inclusion  if  all  constraints  have 
resources  remaining. 

Loulou  and  Michaelides’  approach  (1979)  expands  on  Toyoda’s  (1975)  approach. 
If  two  candidate  projects  have  an  equal  Vj ,  it  is  more  advantageous  to  select  first  that 

project  which  consumes  less  resource.  They  proposed  four  different  ways  to  select  the 
penalty  factor  v . .  To  calculate  the  effective  gradient,  three  important  concepts  should  be 

introduced. 

DA^  +  Qy  :  total  consumption  of  resource  i  if  Xj  is  added  to  the  current  solution 

where  DAi  is  an  amount  of  resource  consumed  so  far. 

1  -  {DA.  +  Qy)  :  amount  of  resource  i  remaining  if  xj  is  added  to  current  solution 

future  potential  demand  for  resource  i  if  Xj  is  added  to  current 
solution  where  SC  is  a  set  of  candidate  variables. 

The  first  choice  of  penalty  factor  Vj  is  then: 

Vj  =Max^^,_^  {(Z)4  +ay  )(^  ^,5^%  “  )  /(I  “  DA,  -a,y)).  (4) 

The  second  method  chooses  penalty  factor  v ^  to  decrease  the  importance  of  the 
ratio  by  taking  its  square  root: 

V,=Max,„ . J(PA^+a.  ■  (5) 
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Third  and  fourth  methods  are  modified  from  the  first  and  seeond  methods, 


respeetively.  These  approaehes  use  v  .  until  Max,  DAi  beeomes  “close  enough”  to  1  and, 

from  then  on,  select  projects  according  to  their  profits  alone.  This  modification  is  called 
a  switch.  The  switch  is  actuated  when  some  resource  becomes  so  scarce  as  to  suggest 
that  the  algorithm  is  close  to  terminating. 

Pirkul  (1987)  and  Glover(1977)  use  a  multiplier  method  and  surrogate  constraints 
to  transform  the  MKP  into  a  knapsack  problem  whose  solution  provides  a  bound  to  the 
original  MKP.  A  surrogate  constraint  is  an  inequality  implied  by  the  constraints  of  an 
integer  programming  problem  (Glover,  1968).  A  surrogate  constraint  is  formed  using  a 
non-negative  linear  combination  of  the  constraints.  The  surrogate  problem  is  defined  as 
follows: 

Z^(//)  =  max  cx 

St.  {juA)x<  /ub  (6) 

e{0,l}  \/  j  ^  N 

where  ju  is  positive  multiplier  vector  of  size  m.  The  best  bound  using  this  scheme  is 
determined  by  locating  a  set  of  multipliers  ju*  such  that 

ZXm)  (V) 

If  jU*  is  known,  then  bounds  from  Z^{ju*)  are  better  than  bounds  from  both  a  LP- 

relaxation  and  a  Lagrangian  relaxation  (Glover,  1968).  Notice  that  problem  (6)  is  simply 
a  0  -  1  knapsack  problem  of  the  form  in  equation  (1). 

Pirkul  (1987)  solves  a  series  of  continuous  knapsack  problems  while  conducting  a 
single  dimensional  search  process  for  each  //,.  Alternatively,  he  could  have  used  the  dual 
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variables  associated  with  each  constraint  in  the  linear  programming  relaxation.  For 
Pirkul’s  approach,  (1)  determine  a  set  of  surrogate  multipliers  using  dual  variables  from 
the  relaxation.  Then  (2)  calculate  Cjl{juA)j  ratios,  re-indexing  decision  variables 

according  to  the  decreasing  order  of  these  ratios.  And  (3),  sequentially  fix  variables  to  1, 
considered  in  index  order,  retaining  problem  feasibility.  Denote  this  solution  and 
define  -  {j  \  x.  =  1}  as  the  set  of  variables  set  to  1.  Finally,  (4)  for  each  j^S\  setx,-  =  0 

and  repeat  (3)  to  define  a  new  feasible  solution  that  temporarily  excludes  an  xj  from 
consideration. 

Osorio  et  al.  (2002)  improved  on  Glover’s  approach  by  introducing  cutting  and 
surrogate  analysis.  They  fix  some  variables  to  zero  and  separate  the  rest  into  two  groups, 
those  that  tend  to  be  zero  and  those  that  tend  to  be  one  in  an  optimal  solution.  Using  an 
initial  feasible  solution,  they  generate  logic  cuts  based  on  analysis  before  solving  the 
problem  using  branch-and-bound.  The  dual  surrogate  constraint  provides  a  useful 
relaxation  of  the  constraint  set,  and  can  be  paired  with  the  objective  function.  The 
resulting  surrogate  has  the  following  property: 

(8) 

j&Ni&M+N  ieM-\-N 

where  u  is  the  surrogate  multiplier  vector.  Since  Z  M,  h,  is  the  continuous  LP  relaxed 
solution,  this  is  the  current  upper  bound  (UB). 

Now,  constraint  pairing  is, 

Y,CjXj  >  LB  (9) 

JgN 

X(  ^u,a,j)x^.<UB  (10) 

jeNieM+N 
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Defining  S'  as  a  set  that  eontains  the  indiees  of  x  variables  whose  values  in  the 
relaxed  LP  solution  are  equal  to  zero,  and  making  =  Sj ,  the  resulting  eombined 

ieM+N 

eonstraint  ean  be  expressed  as 

Y,{Sj-Cj)x^<UB-LB.  (11) 

Constraint  (11)  makes  the  bounds  on  the  eomponents  of  x  stronger.  Apparently, 
the  value  of  sj  -  cj  is  larger  than  the  value  of  UB  -  LB,  the  eorresponding  xj  must  be  zero. 
UB  eannot  be  ehanged  beeause  it  is  the  solution  from  an  LP  relaxation  of  the  problem. 
Thus,  if  we  get  a  better  LB,  we  ean  inerease  the  number  of  integer  variables  fixed  to  zero. 
This  deereases  the  number  of  variables  to  be  eonsidered  in  the  braneh-and-bound 
algorithm. 

2,3,3  Experimental  comparison  of  heuristics  for  the  knapsack  problem 

General  greedy  algorithms  for  MKPs  usually  give  different  answers  based  on  the 
problem  eharaeteristies.  Zanakis  (1977)  eompared  three  heuristie  methods  (Senju- 
Toyoda  (1968),  Koehenberger  et  al.  (1974),  and  Hillier  (1969)).  For  the  eomparison,  he 
ereated  a  set  of  randomly  generated  0-1  test  problems  with  nonnegative  eoeffieients  and 
also  used  benehmark  test  problems.  Zanakis  eontrolled  the  number  of  variables  (V),  (20, 
60  ,100,  200,  500  and  1000),  the  number  of  oonstraints(C),  (20,  60,  100  and  1000),  and 
the  degree  of  eonstraint  slaekness  (30%,  50%,  and  90%)  in  the  test  problems.  He 
measured  eomputer  running  time,  error  and  relative  error  on  the  test  problems.  Zanakis 
results  suggest  all  three  methods  have  solution  times  that  inerease  linearly  up  to  40-50 
variables  and  200  eonstraints  but  exponentially  thereafter,  and  mueh  faster  with  respeet  to 
the  number  of  variables  (F)  than  eonstraints  (Q.  Hillier’s  (1969)  algorithm  was  the  most 
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accurate  but  much  slower  than  the  other  two.  Koehenberger’s  et  al.  (1974)  heuristie  was 
the  fastest  of  the  three  with  tight  eonstraints  and  the  most  aceurate  with  loose  eonstraints. 
In  general  the  Senju-Toyoda  (1968)  algorithm  was  the  fastest,  but  least  aecurate  on  small 
and  medium  size  problem.  Therefore,  Zanakis  suggested  seleeting  the  best  heuristies 
based  on  the  problem  eharaeteristies.  Loulou  and  Miehaelides  made  a  similar  suggestion 
based  on  their  researeh  results  (1979). 

Fox  and  Naehtsheim  (1990)  evaluated  six  greedy  seleetion  rules  on  zero-one 
knapsack  problems.  They  used  six  rules  (I  through  VI)  aeeording  to  how  to  choose  the 
penalty  faetor  {wj).  The  importanee  of  the  projeet  (g,)  ean  be  said  to  be 
gj  =Cj  / Qjj  where  wj  represents  a  “  weight”  assigned  to  eonstraint  i  refleeting  its 

“importanee”  or  the  relative  “seareity  of  resouree  i  ”.  A  greedy  algorithm  seleets  Xi,  and 
sets  it  to  1  aeeording  to  the  largest  gj .  Rules  I,  II  and  III  are  based  on  Fox  and  Scudder 
(1985).  For  Rule  I,  let  wf  and  bf  be  “the  weight  assigned  to  eonstraint  i  and  the  amount 

of  resource  i  remaining  just  prior  to  the  pih  applieation”,  and  let  67^^  =  min.^^jh,^}  where 
C  is  the  set  of  eonstraints.  Note  that  67^^  >  0 .  Rule  I  is  as  follows: 

[0,  Otherwise 

Rules  II  and  III  are  modified  from  Rule  I.  A  eonstraint  tightness  eonsideration  is 
introduced  in  these  rules.  Let  sfhe  the  tightness  of  eonstraint  i  prior  to  the  p\h 

applieation.  In  Rule  II,  sf  =b[  !  ,  and  in  Rule  III,  sf  =  bj’  -  where  V  is 

the  set  of  indiees  of  variables  that  are  possible  candidates  for  setting  to  one. 
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The  weights  used  are  as  follows: 


|l, 

[O,  Otherwise 


(13) 


for  all  i  e  C,  where  }  . 

Rule  IV  is  known  as  simple  greedy  in  that  all  constraints  are  regarded  as  equally 
important  and  are  given  an  equal  penalty  factor.  Rule  V  and  VI  are  based  on  Toyoda 
(1975).  For  their  empirical  analysis,  Fox  and  Nachtsheim  varied  four  parameters  on  1440 
randomly  generated  test  problems.  Their  parameters  were  number  of  variables,  number 
of  constraints,  constraint  matrix  density,  and  slackness.  They  measured  the  average 
relative  efficiencies  and  the  average  rank  of  the  objective  function  among  all  six  rules. 
They  suggested  that  Rule  IV  seems  to  be  the  best  algorithm  in  terms  of  relative 
efficiencies  and  rank  of  the  objective  function.  However,  in  the  mixed  slackness 
(tightness  value  Si  =  0.3  with  probability  0.5  and  Si  =  0.7  with  probability  0.5)  problems. 
Rule  I,  from  Fox  and  Scudder  (1985)  was  superior  because  Rule  IV  uses  the  same 
importance  and  same  weight.  Also,  they  concluded,  “  The  simplest  rule  is  the  best, 
except  when  the  constraints  exhibit  mixed  slackness.” 

2,3,4  Studies  into  effects  of  correlation  on  solution  procedure  performance 

Some  MKPs  can  be  quickly  solved  even  if  n  is  very  large,  while  other  problems 
cannot  be  easily  solved  for  n  equal  to  a  few  hundred.  One  reason  may  be  that  the 
correlation  between  objective  function  and  each  set  of  constraint  coefficients,  and  the 
correlation  between  constraint  coefficients  effects  solution  procedure  performance. 

Many  authors  developed  their  randomly  generated  data  set  to  verify  their  algorithm,  but 
few  have  actually  studied  the  effects  of  correlation  among  the  test  problem  coefficients. 
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Martello  and  Toth  (1988,  1997)  devised  three  classes  of  correlation  to  check 
computational  performance  for  knapsack  problems: 

Uncorrelated:  wj  uniformly  random  in  [  1 ,  a] , 

Pj  uniformly  random  in  [1,  a], 

Weakly  Correlated:  wj  uniformly  random  in  [1,  a  ], 

Pj  uniformly  random  in  [  wj  -  5,  iv,  +  5  ] 

Strongly  Correlated:  wj  uniformly  random  in  [1 ,  a], 

Pj=Wj+^ 

where  pj  =  profit  of  item  j,  and  wj  =  weight  of  item  j  given  n  items,  and  a  and  5  are 
prefixed  constants  (  Martello  and  Toth,  1997).  These  induction  strategies  have  been 
widely  used  in  empirical  studies  despite  the  fact  that  the  weakly  correlated  scheme 
produced  coefficients  correlated  to  a  value  of  0.98. 

Martello  and  Toth  (1988)  conducted  experiments  with  their  exact  algorithm  MT2 
solving  problems  with  each  correlation  level  with  up  to  more  than  100,000  variables. 
They  reported  uncorrelated  and  weakly  correlated  instances  were  easily  solved.  However 
the  strongly  correlated  instances  were  very  difficult  to  solve.  They  could  be  solved  for 
small  number  of  variables  and  constraints,  using  a  dynamic  programming  algorithm  that, 
however,  would  not  work  on  larger  instances  due  to  excessive  space  and  time 
requirements.  In  short,  the  results  of  Martello  and  Toth  for  the  zero-one  knapsack 
problem  indicate  that  problems  with  nearly  perfectly  positive  correlated  processing  times 
and  weights  are  significantly  harder  than  the  uncorrelated  problems. 

Hill  and  Reilly  (2000)  measured  how  the  coefficient  correlation  structure  affects 
solution  performance  using  randomly-generated  test  sets.  For  test  sets,  they  controlled 
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these  problem  generation  parameters:  correlation  measure  (Pearson  or  Spearman), 
correlation  structure,  and  the  constraint  slackness: 

Let  U{1,  2,  . . .,  40}  be  the  random  variable  representing  the  values  of 
coefficients  in  the  first  constraint,  let  A  ~U{1,2,  ...,  15}  be  the  random  variable 
representing  the  values  of  the  coefficient  in  the  second  constraint,  and  let  C~  U  { 1 ,  2,  . . . , 
100}  be  the  random  variable  representing  the  values  of  the  objective  function  coefficients 
in  2KP.  The  three  correlation  terms  in  the  correlation  structure  of  2KP  are  , 

and  p.  ^2  with  p  =  ( p  ^  ,  p  ^  ,p,  2  )•  A  “  slackness”  measure  for  constant  i,  Si,  is 

defined  as  the  ratio  of  the  right-hand  side  coefficient  in  constraint  i  to  the  sum  of  the 
coefficients  in  that  constraint.  Two  levels  of  slackness  are  examined  in  this  study:  Si  = 
0.30,  0.70,  i=  1,2.  Each  of  the  four  possible  setting  of  5*1  and  S2  is  referred  to  as  a 
constraint  slackness  setting  (Hill  and  Reilly,  2000). 

For  their  study,  they  used  CPLEX  and  Toyoda  (1975)  as  solution  methods.  Their 
goal  was  to  investigate  how  problem  structure  effects  solution  procedure  performance  by 
either  algorithm  (CPEEX)  or  heuristic  (TOYODA).  They  measured  the  number  of  nodes 
for  CPEEX  performance  and  the  relative  error  for  the  TOYODA  performance.  Between 
Pearson  and  Spearman  test  problems,  Spearman  correlation  problems  were  harder.  For 
correlation  structure,  they  found  that  the  difficult  problems  requiring  more  nodes  have 
larger  differences  between  y0^^2,and  p.^2  in  CPEEX.  The  negative  values  of 

/?^,^2  yields  the  harder  problems  for  TOYODA.  Interestingly,  the  challenging  problems 

for  CPEEX  were  easy  for  TOYODA  as  it  found  optimal  solutions.  For  constraint 
slackness,  tight  constraints  provide  more  challenging  problems  for  both  CPLEX  and 
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TOYODA.  The  interaction  between  correlation  structure  and  constraint  slackness  is  that 


tighter  constraints  and  constraint  coefficients  with  a  wider  range  of  values  produce  more 
difficult  problems.  However,  in  CPLEX,  positive  interconstraint  correlation  usually 
yields  more  simple  problems.  For  TOYODA,  tight  constraints  and  the  negative  value  of 
make  problems  harder  to  solve.  Their  results  indicate  that  an  algorithms’ 

performance  depends  on  the  problem  “characteristics”. 
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III.  Methodology 


3.1  Introduction 

The  heuristic  methods  proposed  by  Toyoda,  Senju-Toyoda  (S  -  T)  and  Loulou  - 
Michaelides  (L  -  M)  are  examined  in  this  section.  The  Test  problem  characteristics  in 
the  library  section  describes  the  test  problems  characteristics  of  those  problems  solved  by 
each  heuristic  method.  The  Approaches  to  the  empirical  analysis  of  algorithms  and 
heuristics  section  discusses  general  heuristic  approaches  to  the  empirical  analysis  of 
algorithms  and  heuristics.  The  final  sections  discuss  how  to  analyze  comparative  data 
and  how  to  compare  heuristic  performance. 

3.2  Test  problem  characteristics  in  the  library 

Our  objective  is  to  examine  heuristic  performance  as  a  function  of  constraint 
tightness  and  problem  correlation  structure.  We  used  the  Spearman  2KP  problems 
developed  and  used  by  Hill  and  Reilly  (2000).  For  each  problem,  the  number  of 
constraints  is  2  (i.e.,  the  2KP),  and  the  number  of  variables  is  100.  These  test  problems 
were  created  using  a  Spearman  rank  correlation  induction  method.  This  method  creates 
values  of  trivariate  random  variables  to  represent  the  coefficients  {cj,  aij,  a2j),  and  ensure 
the  sets  of  values  have  the  desired  correlation  structure.  The  objective  function 
coefficients,  cj,  are  integer  numbers  uniformly  distributed  from  1  to  100.  The  coefficients 
of  the  first  constraint,  ajj,  are  integer  numbers  uniformly  distributed  from  1  to  25  while 
the  coefficients  of  the  second  constraint,  a2j,  are  integer  numbers  uniformly  distributed 
from  1  to  40.  The  three  correlation  terms  are  ,  p„^2 ,  and  p.^2  ■  The  terms  and 

A  A  i^A 
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Pqa^  represent  the  eorrelation  between  objeetive  funetion  eoeffieients  (9)  and  eonstraint 

eoeffieients  (a;y  and  a2;).  The  term  represents  the  eorrelation  between  the  two 

eonstraint  eoeffieients.  The  range  of  eorrelation  levels  for  eaeh  eorrelation  term  are  set  as 
follows: 


e  {  -  0.99997,  -  0.49999,  0,  0.49999,  0.99997} 
e  {  -  0.99773,  -  0.49887,  0,  0.49887,  0.99773} 
e  {  -  0.99752,  -  0.49876,  0,  0.49876,  0.99752} 


Within  eaeh  set  of  eorrelation  values,  the  largest  absolute  values  represent  the 
extreme  eorrelation  level.  Considering  eaeh  possible  eombination  of  eorrelation  value 
implies  125  eombinations.  However,  of  these  125,  only  45  represent  positive  definite 
eorrelation  matriees.  For  eorrelation  strueture,  the  five  levels  of  eorrelation  for  eaeh 
eorrelation  term  are  eoded  as  {-2,  -1,0,  1,2}. 

For  eonstraint  slaekness,  two  different  eonstraint  slaekness  are  examined  in  this 
study.  Slaekness  number  of  1  indieates  a  slackness  of  0.30  and  a  slackness  number  of  2 
indicates  a  slackness  of  0.70.  The  right-hand  side  coefficients  {hi)  are  set  using  the 
relation: 

b,=S,Y,aij 

J^N 

where  =  0.30  and  S2  =  0.70. 

Each  of  the  four  possible  setting  of  Si  and  S2  is  referred  to  as  a  constraint 
slackness  setting  (0.30  or  0.70).  We  have  a  total  of  1 120  problems  with  224 


19 


combinations  of  45  feasible  correlation  struetures,  four-eonstraint  slaekness  settings,  and 
5  replications  each. 

3,3  Approaches  to  the  empirical  analysis  of  algorithms  and  heuristics 

Barr  et  al.  (1995)  outline  a  general  approaeh  for  conducting  empirical  testing  of 
heuristies.  We  follow  the  guidelines  for  the  empirical  testing  of  heuristics  by  Barr  et  al: 

1.  Define  the  goals  of  the  experiment: 

My  goal  is  to  conduct  a  rigorous  computational  study  to  isolate  and  examine  the 
performanee  of  three  greedy  heuristies,  TOYODA,  Senju  -  Toyoda,  Loulou  - 
Miehaelides  based  on  eonstraint  slaekness  and  correlation  strueture.  The  purpose  of  this 
study  is  to  conduct  a  computational  test,  gain  insight  into  how  eorrelation  strueture  and 
eonstraints  slaekness  affeet  three  different  heuristies,  and  develop  a  new  heuristie  based 
on  the  results.  In  other  words,  which  heuristic  method  yields  the  best  solution  under 
certain  correlation  and  slaekness  conditions  and  how  might  we  take  advantage  of  this 
knowledge. 

2.  Choose  measures  of performance  and  factors  to  explore: 

A  faetor  is  any  eontrollable  variable  in  an  experiment  that  effeets  the  outcome  of 
the  experiment.  The  faetors,  in  this  experiment,  are  the  four-eonstraint  slackness 
(combination  of  tight  and  loose  constraints)  levels  and  45  feasible  eorrelation  structures. 

A  measure  is  the  outeome  of  an  experiment.  All  three  heuristies  were  eoded  in 
Visual  Basie  for  Applications  within  Excel  and  run  against  the  problems.  Eaeh 
combination  of  constraint  slackness  and  eorrelation  yields  a  best  heuristie  method  and  by 
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counting  the  number  of  times  eaeh  method  is  best  for  various  faetor  eombinations,  we 
ean  measure  whieh  heuristie  is  best  for  eaeh  problem  strucuture. 

3.  Design  and  execute  the  experiment: 

As  mentioned  in  Seetion  3.2,  we  use  thel  120  test  problem  set  from  Hill  and 
Reilly  (2000).  These  problems  represent  a  full-faetorial  design  in  eonstraint  slackness 
and  all  feasible  eombinations  of  the  eorrelation  value  sets. 

4.  Analyze  the  data  and  draw  conclusions: 

Barr  et  al.  suggested  that  there  were  at  least  three  sourees  of  variation  one  must 
reeognize.  These  are  as  follows:  (1)  variation  among  algorithm  performanee,  (2) 
variation  due  to  problem  parameters  and  (3)  variation  within  problems.  Based  on  Hill 
and  Reilly  (2000),  (3)  is  negligible.  We  therefore  foeus  on  (1)  and  (2)  with  emphasis  on 
(1)  sinee  we  hope  to  gain  insight  into  why  eertain  heuristies  do  well  or  not  so  well  on 
eertain  problems. 

3,4  Computer  coding  of  three  heuristic  methods 

In  order  to  eliminate  the  possible  influences  eaused  by  any  type  of  restriction, 
each  constraint  is  normalized  by  dividing  all  eonstraint  row  eoeffieients  (a,y)  by  its  right- 
hand  side  eoefficient  (h/),  yielding  a  problem  with  eaeh  h,  =  1 . 

For  TOYODA’s  heuristie,  we  eoded  the  original  primal  effeetive  gradient 
method.  Toyoda  (1975)  ineluded  origin  -  moving  in  his  primal  effeetive  gradient 
method,  but  we  did  not  use  this.  Let  Fij  be  the  normalized  eonstraint  eoeffieients  (a,y). 
When  eumulative  quantity  veetor  is  a  zero  veetor  sueh  as  in  the  first  iteration,  the 
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.  Let  Pj  be  a  vector  of  {Fij,  F2j 


gradient  for  each  variable  is  calculated  as  Gj  = 


).  For  most  iterations,  the  effective  gradient  is 


Cj  -42 

iF,,+F,j) 


_^Fn 

PpPu 


where  Pu  is  cumulative  total 


quantity  vector,  i.e.,  the  vector  of  used  resources.  Variables  are  added  according  to  best 
effective  gradient  until  the  constraint  resources  are  fully  used. 

For  S  -  T  heuristic,  we  coded  the  improved  method  suggested  by  the  authors. 

The  improved  method  also  normalizes  constraint  coefficients.  The  following  vectors  are 
introduced:  Ai  =  vector  of  constraint  coefficients  which  is  normalized,  Ai  =  (an,  aa).  R  is 
the  sum  vector  of  Ai.  B  is  the  vector  of  right  -  hand  side  values.  Since  RHS  is 
normalized,  B  =  {\,\).  5*  is  slackness  vector.  The  following  vector  equations  hold: 

R  =  Ay  +  A^  H  +  ^100 


S  =  R-B 


c-  - 

The  gradient  is  calculated  as:  Gj= — —  .  Then,  since  S  -  T  is  a  dual  approach,  S 

PjS 

-  T  drops  variables  according  to  best  effective  gradient  until  feasibility  is  achieved. 

As  mentioned  in  section  2.3.2,  for  the  L  -  M  heuristic,  Loulou  and  Michaelides 
(1979)  suggested  four  different  methods  to  find  the  penalty  vectors.  We  use  method  Ml 
to  define  the  penalty  vector.  That  is: 

Vj  =  100 {(Z)4.  +  a.. “  «/,■ ) /(I “ ^4  “  )} 

where  DAi  is  an  amount  of  resource  consumed  so  far. 
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Notice  L  -  M  heuristic  picks  the  worst  penalty  eost  (Fj)  between  two  eonstraints. 


The  effeetive  gradient  is  ealeulated  as: 


L  -  M  adds  variables  aeeording  to  the 


effective  gradient  until  eonstraints  are  met. 

All  three  heuristie  eodes  are  available  in  Appendix  A,  B  and  C,  respectively. 
Some  representative  results  are  shown  on  Table  1.  All  problems  were  sueeessfully 
solved. 


Table  1.  Results  by  Three  Heuristics  Solutions  over  1120  problems 


Prob 

Num 

Rep 

Num 

Cl 

Slack 

C2 

Slack 

CA1 

CA2 

A1A2 

IP 

TOYODA 

S-T 

L-M 

1 

1 

1 

1 

2 

2 

2 

1480 

1468 

1468 

1468 

2 

2 

1 

1 

2 

2 

2 

1644 

1631 

1614 

1584 

3 

3 

1 

1 

2 

2 

2 

1497 

1454 

1454 

1441 

4 

4 

1 

1 

2 

2 

2 

1704 

1694 

1696 

1603 

5 

5 

1 

1 

2 

2 

2 

1619 

1597 

1597 

1549 

6 

1 

1 

2 

2 

2 

2 

1647 

1626 

1635 

1611 

7 

2 

1 

2 

2 

2 

2 

1787 

1772 

1770 

1594 

8 

3 

1 

2 

2 

2 

2 

1590 

1586 

1585 

1552 

9 

4 

1 

2 

2 

2 

2 

1629 

1627 

1628 

1461 

10 

5 

1 

2 

2 

2 

2 

1669 

1663 

1665 

1574 

11 

1 

2 

1 

2 

2 

2 

1732 

1722 

1725 

1647 

1117 

2 

2 

2 

2 

-2 

-2 

3574 

3540 

3535 

3537 

1118 

3 

2 

2 

2 

-2 

-2 

3683 

3623 

3620 

3640 

1119 

4 

2 

2 

2 

-2 

-2 

3812 

3544 

3435 

3532 

1120 

5 

2 

2 

2 

-2 

-2 

3696 

3648 

3631 

3646 

Prob  Num  =  Problem  Number 

Rep  Num  =  number  of  replications 

Cl  Slack  =  slackness  of  first  constraint  (  1  =  0.3,  2  =  0.7) 

C2  Slack  =  slackness  of  second  constraint  (1  =  0.3,  2  =  0.7) 

CA1  =  correlation  between  c,and 

{  -2  =  -0.99997,  -1  =  -0.49999,  0  =  0,  1  =  0.49999,  2  =  0.99997  ) 
CA2  =  correlation  between  c,and  a2j 

(  -2  =  -0.99773,  -1  =  -0.49887,  0  =  0,  1  =  0.49887,  2  =  0.99773  ) 
A1A2  =  correlation  between  ayand  a2j 

(  -2  =  -0.99752,  -1  =  -0.49876,  0  =  0,  1  =  0.49876,  2  =  0.99752  ) 
IP  =  Integer  optimal  solution  (or  best  known) 

TOYODA  =  solution  by  Toyoda’s  heuristic 

S-T  =  solution  by  Senju  -  Toyoda’s  heuristic 
L-M  =  solution  by  Loulou  -  Michaelides’  heuristic 
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3,5  Analysis  of  Results 


3,5.1  Relative  Error 

The  ultimate  goal  of  a  heuristic  is  to  find  an  optimal  solution.  Short  of  this,  the 
heuristic  should  come  near  the  optimal.  Since  we  know  the  true  optimum  or  best  known 
solution  for  each  test  problem,  we  can  use  the  relative  error  measure,  that  is,  the  smallest 
relative  error  provides  the  closet  solution  to  the  optimum.  Let  Z,  be  the  value  of  objective 
function  obtained  by  heuristic  i  where  i  =  Toyoda,  S-T  or  L-M  and  Z*  be  the  optimal  or 
best  known  solution.  Then  the  relative  error: 

=  100-(Z*-Z.)/Z* 

Based  on  the  relative  error,  we  choose  the  best  method  excluding  ties  under 
certain  correlation  and  constraint  slackness.  Since  our  goal  is  to  know  why  certain 
correlation  and  constraint  slackness  levels  make  specific  methods  perform  well  (i.e.,  the 
smallest  relative  error),  we  count  the  number  of  times  a  heuristic  is  the  best,  excluding 
the  number  of  ties,  by  correlation  structure  and  constraint  slackness  settings.  This  is 
shown  Table  2. 


Table  2.  Relative  Error  of  Three  Heuristics 


Problem  No, 

TOYODA 

S-T 

L-M 

Best  Method 

1 

0.810811 

0.810811 

0.810811 

Tie 

2 

0.790754 

1.824818 

3.649635 

TOYODA 

3 

2.872411 

2.872411 

3.740815 

Tie 

4 

0.586854 

0.469484 

5.92723 

S-T 

5 

1.358863 

1.358863 

4.323657 

Tie 

6 

1.275046 

0.728597 

2.185792 

S-T 

7 

0.839396 

0.951315 

10.80022 

TOYODA 

8 

0.251572 

0.314465 

2.389937 

TOYODA 
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3,5.2  Chi-square  test 

We  wish  to  know  whether  or  not  there  is  a  best  method  among  the  three  heuristics 
over  each  correlation  and  constraint  slackness.  We  use  a  Chi-square  test  to 
determine  whether  or  not  any  heuristic  method  is  significantly  better  than  the  other 
methods. 

Ho ;  Methods  do  not  differ. 

Hi  :  At  least  2  methods  differ. 

To  compute  the  chi-square  test,  first  divide  the  entire  range  of  the  fitted 
distribution  into  3  equal  intervals  [ao,  ai),  [ai,  a2),  [a2,  as)  where  [ao  ai)  represents 
Toyoda’s  heuristic,  [ai,  a2)  represents  Senju-Toyoda’s  heuristic,  and  [a2,  as)  represents 
Loulou  -  Michaelides’s  heuristic.  Then  we  have 

Nj  =  number  of  times  best  in  the  interval  [a,_;,  a,),  for  j  =\  ,  2,  3 
(Note  that  1  =  Toyoda,  2  =  S-T,  3  =  L-  M  heuristic) 

Next,  we  compute  the  expected  proportion,  j:?,,  of  the  N/s  that  should  fall  in  the yth 
interval.  Since,  under  Ho  we  assume  that  there  is  no  difference  in  the  three  methods,  we 
can  define  pj=  1/3.  Finally,  the  test  statistic  is 

Since  npj  is  the  expected  number  of  times  the  best  j  heuristic  occurs,  if  Ho  is  true, 
we  would  expect  ^  to  be  small  if  the  fit  were  good.  We  reject  Ho  if  ^  is  too  large.  For 
this  test,  a  =  0.1  is  used,  so  the  critical  value  is  ^ o.oi.d.f.. 
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3,5,3  Sign  test  for  paired  samples 

We  use  a  sign  test  to  determine  whether  or  not  one  heuristic  outperforms  another. 
For  a  sign  test,  the  hypothesis  is  as  follows: 

Ho:  two  heuristics  are  identically  distributed. 

Hi:  two  heuristics  are  statistically  different. 

If  Ho  is  true,  then  for  any  test  problem  either  heuristic  has  an  equal  chance  of 
being  the  best.  Therefore,  the  distribution  of  outcomes  has  the  Binomial  distribution  B(n, 
0.5).  Let  f/be  the  number  of  times  the  first  heuristic  is  best.  If  Ho  is  true,  then  U  ^  B{N, 

0.5)  and  is  approximated  by  a  normal  distribution  having  mean  ju  =  N  x  and  standard 


deviation  a  =  J  N  x  .  To  find  the  significance  level  of  the  result,  we  calculate  : 


P{X  >  [/)  «  PiX  >U-  0.5)  =  P 


Z> 


U-0.5-^ 


(j 


Since  this  is  a  two-tailed  test,  and  the  normal  distribution  is  symmetric,  we  use  an 
absolute  value  on  the  critical  value.  We  use  a  =  0.05  level  of  significance  to  decide 
whether  to  fail  to  reject  Ho  or  reject  Ho. 


3,6  New  heuristic 

3,6,1  New  Combined  Heuristic 

We  will  study  which  heuristic  has  the  best  solution  under  various  conditions,  such 
as  different  constraint  slackness  settings,  correlation  structures  or  a  combination  of  both. 
In  other  words,  which  heuristic  under  what  conditions  becomes  statistically  the  best 
heuristic.  We  use  the  condition  which  made  one  of  the  three  heuristics  the  best  heuristic 
for  a  new  combined  heuristic.  That  is,  we  first  investigate  the  characteristics  of  a 
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problem  as  to  whether  it  is  dominated  by  constraint  slackness,  correlation  structures,  or  a 
combination  of  both.  Then,  we  choose  a  heuristic  that  is  likely  to  produce  the  closest 
solution  to  the  best-known  solutions  among  the  three  heuristics,  TOYODA,  S  -  T,  and 
L-M. 

3,6.2  Cho  Heuristic 

After  studying  why  the  best  heuristic  works  well  and  why  the  poorer  heuristic 
does  not  work  well  in  Chapter  IV,  we  combine  the  merits  of  the  three  heuristics  into  one, 
new  heuristic.  Since  three  heuristics  use  different  penalty  factors  as  well  as  feasibility, 
for  example,  TOYODA  and  L-M  start  from  an  empty  solution  and  maintain  feasibility 
while  S  -  T  starts  from  an  infeasibility  region,  we  may  predict  that  some  other  factors  are 
influential  in  creating  a  better  solution.  Therefore,  we  develop  a  new  heuristic  that 
includes  the  favorable  influential  factors. 
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IV,  Analysis  of  Legacy  Heuristics 


This  chapter  presents  the  results  of  an  empirieal  study  of  the  Senju  -  Toyoda 
(1968),  Toyoda  (1975),  and  Loulou  -  Michaelides  (1979)  heuristics  as  applied  to  the  2KP 
test  problem  set  of  Hill  and  Reilly  (2000).  The  purpose  of  this  study  was  to  gain  insight 
into  heuristic  performance  to  build  new  heuristies  for  MKPs. 


4.1  Constraints  slackness 

The  Results  section  deals  with  the  performance  of  three  heuristics  over  four 
different  combinations  of  constraint  slackness  (tight,  loose).  Most  of  the  analysis  results 
present  data  as  counts  of  best  performer  (excluding  ties)  among  the  three  heuristics. 
Graphics  are  validated  using  non-parametrie  statistical  test. 

4.1.1  Results 

The  overall  performance  of  the  three  heuristics  is  summarized  in  Table  3  and 
Figure  1.  The  data  reflect  the  various  combinations  of  constraint  slackness  coded  as; 

1  =  0.3  and  2  =  0.7. 


Figure  1 .  Counts  of  best  of  three  heuristic  methods  under  constraint  siackness 
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Toyoda  has  long  been  the  benchmark  greedy  heuristic  for  MKP  problems.  These 
results,  however,  suggest  that  S  -  T  might  be  better  in  some  cases  and  L  -  M  better  in 
other  cases.  The  counts  provided  in  Table  3  appear  convincing;  there  is  a  difference  in 
heuristic  performance. 


Table  3.  The  number  of  times  best  by  each  heuristic  under  constraint  slackness 


Constraint 

Slackness 

Method 

Toyoda 

S-T 

L-M 

1,1 

87 

15 

123 

1,2 

22 

186 

15 

2,1 

24 

184 

13 

2,2 

41 

17 

147 

If  there  were  no  differences  in  performance  of  these  heuristics,  we  would  expect 
each  row  in  Table  3  to  have  fairly  equal  counts.  A  chi-square  test  of 

Ho:  All  heuristics  equal 
Hi:  Heuristics  differ 

is  summarized  in  Table  4.  These  results  confirm  the  intuition  from  Table  3;  for  each 
slackness  setting  there  is  a  preferred  heuristic.  The  following  sections  examine  each 
slackness  setting  in  detail. 

Table  4.  Chi-square  test  for  constraint  slackness 


Constraint 

Slackness 

df 

Probability 

Reject  Region 
(a  =  0.1) 

1,1 

80.64 

2 

3.08E-18 

Reject  Ho 

1,2 

251.9552 

2 

1.94E-55 

Reject  Ho 

2,1 

248.6968 

2 

9.91E-55 

Reject  Ho 

2,2 

140.0585 

2 

3.86E-31 

Reject  Ho 
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4,1.2  Analysis  of  constraint  slackness  of  Si  =  0,3  and  S2  =  0,3  (1,1) 

Table  3  and  Figure  1  indicate  that  TOYODA  and  L  -  M  yield  better  solutions 
than  S  -  T  when  constraints  are  tight.  The  question  is  whether  TOYODA  and  L-  M 
differ.  A  sign  -  test  is  used  to  examine  this  question.  Table  5  summarized  the  sign  test 
indicating  there  is  no  difference  between  TOYODA  and  L  -  M  for  (Si,  S2)  =  (0.3,  0.3) 
problems. 


Table  5.  Sign  test,  TOYODA  vs.  L  -  M  under  Si  =  0.3  and  S2  =  0.3 


No,  of  Non-Ties 

No,  of  Ties 

TOYODA  Better 

L  -  M  Better 

272 

8 

168 

124 

Ho 

Two  heuristics  are  identically  distributed. 

Normal 

Approximation 

5(272,  0.5) 

P(A>148)«P(Z>  0.08157 

8.2462 

Since  this  is  a  two-tailed  test,  the  value  17=  168  would  be 
significant  at  a  level  of  0.1631 

Rejection  Region 

(a=  0.05) 

Fail  to  reject  Ho 

The  challenge  is  understanding  why  TOYODA  (and  L  -  M)  beats  S  -  T  for  these 
problems.  Toyoda’s  heuristic  is  a  primal  effective  method.  The  penalty  function  creates 
a  single  effective  gradient  number  based  on  two  limited  resources.  The  penalty  cost  was 
introduced  as  follows: 


f/,-  = 


_iPrPu) 


where  Pi  is  the  vector  {an,  a2i)  and Pu  is  a  cumulative  total  resource  used  vector.  Recall 


Pl¬ 
an  and  a2i  are  normalized  coefficients  in  each  constraint,  i.e.,  a^  =  where  Fij  is  the 


original  coefficient  in  constraints.  Therefore,  Ut  depends  on  the  direction  of  Pu  and  has 
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no  relationship  to  its  magnitude.  To  pick  the  next  new  variable  to  add  among  non- 


selected  variables,  TOYODA  calculates  an  effective  gradient 
Senju  -Toyoda’s  heuristic,  the  penalty  is  as  follows: 


G  =  ^ 


V 


U. 


•  J 


However,  for 


U 


J 


(Pj-S) 

\s 


where  Pi  is  the  vector  {an,  a2i),  S'  is  a  slack  variable  vector,  and  a;,  and  a2i  are  equivalent 

F.. 

to  each  normalized  constraint  multiplied  by  100,  i.e.,  a,  =  —  x  100  .  Let  R  be  the  vector 


of  resource  costs  for  constraint  j,  such  as  i?  =  [r.  :  R.  =  sum{a.j),  /  =  1,2  Thus  S  =  i? 


100. 

The  reason  why  TOYODA  and  L  -  M  yield  better  solutions  is  caused  by  Pu. 
When  the  value  S  is  chosen,  S  does  not  change  at  all  in  Senju-Toyoda’s  method,  while  Pu 
varies  at  each  iteration  in  Toyoda’s  heuristic.  Although  S  always  provides  a  direction 
into  the  feasible  region,  S  is  not  effective  because  it  gives  constraints  equal  weight  and 
goes  too  deep  into  the  feasible  region.  This  causes  more  resources  to  remain.  However, 
the  TOYODA  heuristic  evaluates  Pu  at  each  iteration  after  selecting  a  new  variable.  This 
means  there  are  less  remaining  resources  at  each  constraint.  Thus,  Toyoda  selects  more 
variables  because  it  uses  constraint  resources  effectively.  The  Loulou-Michaelides 
heuristic,  an  extension  of  Toyoda’s  yields  similar  results. 

4.1,3  Analysis  of  mixed  slackness,  Si  ^  S2,  (1,2)  or  (2,1) 

Recall  from  Figure  1  the  overwhelming  advantage  of  S  -  T  when  slackness  levels 
are  mixed.  Since  we  normalized  constraint  coefficients,  the  coefficients,  ay,  represent  a 
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percentage  of  the  right-hand  side  resource.  Senju  and  Toyoda  (1968)  even  suggest  that 
better  solutions  may  be  obtained  when  the  right-hand  side  values  differ  greatly. 

To  understand  why  S  -  T  performs  well,  consider  problems  when  a  (Si,  S2)  = 

(0.3,  0.7).  Since  the  first  constraint  is  tight  and  the  second  constraint  is  loose,  and  S  is 
slack  variable  vector  (si,  S2),  s;  should  be  larger  than  S2. 

Figure  2,  adapted  form  Senju  and  Toyoda  (1968),  depicts  the  S  -  T  heuristic 
approach.  Let  the  axes  represents  resource  average  within  each  constraint  so  Li  and  L2 
represent  right  -  hand  side  values  for  each  constraint.  The  point  R  represents  the 
resource  usage  of  the  initial,  infeasible,  point  used  in  the  S  -  T  heuristic.  The  S  -  T 
heuristic  drops  variables  to  force  the  point  R  into  the  feasible  region,  ideally  along  vector 
S. 


Second 

Constraint 

L2 


0  Li  First  Constraint 

Figure  2.  Effective  iength  of  withdrawai 
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The  slope  of  S  is  almost  horizontal  with  respeet  to  first  eonstraint  axis,  as  shown 
in  Figure  2.  This  means  favoring  the  first  eonstraint  over  the  seeond  eonstraint,  which  in 
the  current  case  is  the  tight  constraint.  It  is  preferable  to  drop  variables  whose 
coefficients  are  smaller  compared  to  their  projected  length  on  the  vector  S,  thus  favoring 
feasibility  with  respect  to  the  tight  constraint. 

The  direction  of  slack  variable  vector,  S,  provides  the  proper  direction  into 
feasible  region.  In  other  words,  as  S  -  T  reaches  the  point  of  feasible  region  (L),  it  is 
using  the  resources  of  the  two  constraint  most  effectively.  In  contrast  to  the  S  -  T  dual 
method,  the  TOYODA  and  L  -  M  primal  methods  pick  variables  to  equalize  resource 
usage.  This  approach  does  not  provide  enough  emphasis  towards  the  problem’s  binding 
(tighter)  constraint.  Therefore,  while  S  -  T  in  problems  with  tight  constraints  uses 
constraint  resources  similarly  compared  with  TOYODA  and  L  -  M;  however,  with  mixed 
constraint  slackness,  S  -  T  is  more  effective.  More  effective  use  of  the  binding  constraint 
translates  into  extra  resources  and  thus  more  projects  selected  yielding  better  solutions. 


Figure  3.  Performance  of  3  heuristic  methods  in  terms  of  best  constraint  resource  use 
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The  data  was  examined  to  determine  which  heuristic  makes  best  use  of  each 


constraint  i.e.,  leaves  the  least  amount  of  slack  in  a  constraint.  Table  6  provides  data 
pertaining  to  test  problem  resource  usage;  Figure  3  is  a  graph  of  the  same  data,  when  (Si, 
S2)  =  (0.3,  0.7).  Table  7  and  Figure  4  provide  similar  data  for  (Si,  S2)  =  (0.7,  0.3).  The 
improved  direction  of  S  -  T  focused  on  the  binding  constraint  means  more  effective  use 
of  the  loose  constraint  resource.  As  the  data  shows,  constraint  usage  is  nearly  equal  on 
the  binding  constraint  but  significantly  better  on  the  loose  constraint  for  S  -  T.  This 
equates  to  more  variables  set  to  one  under  S  -  T  which  in  turn  helps  explain  S  -  T’s 
significantly  better  performance  in  terms  of  solution  value  over  TOYODA  and  L  -  M. 


Table  6.  The  number  of  times  least  slack  remained  in  each  constraint  (Excluding  ties) 


TOYODA 

S-T 

L-M 

Tight  Constraint 
(Si  =  0.3) 

no 

68 

77 

Loose  Constraint 
(82  =  0.7) 

25 

186 

62 

Figure  4.  Performance  of  3  heuristic  methods  in  terms  of  best  constraint  resource  use 
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Table  7.  The  number  of  times  least  slack  remained  in  each  constraint  (Excluding  ties) 


TOYODA 

S-T 

L-M 

Loose  Constraint 
(Si  =  0.7) 

24 

183 

68 

Tight  Constraint 
(S2  =  0.3) 

87 

73 

70 

4,1,4  Analysis  of  constraint  slackness  of  Si  =  0,7  and  S2  =  0,7  (2,2) 

In  cases  where  both  eonstraints  are  loose,  TOYODA  and  L  -  M  heuristies  give 
better  results  than  S  -  T.  TOYODA  and  L  -  M  use  a  similar  penalty  faetor  (Vj); 
furthermore,  both  are  primal  effective  gradient  methods  beeause  L  -  M  is  extended  from 
TOYODA.  A  sign  test,  summarized  in  Table  8,  shows  L  -  M  is  statistieally  better  than 
TOYODA  on  these  problems. 


Table  8.  Sign  test,  TOYODA  vs.  L  -  M  under  Si  =  0.7  and  S2  =  0.7 


No,  of  Non-Ties 

No,  of  Ties 

TOYODA  Better 

L-M  Better 

266 

14 

113 

153 

Ho 

Two  heuristics  are  identically  distributed. 

Normal 

Approximation 

5(266,  0.5) 

112  5  - 133 

P(Y>113)«P(Z>  )  =  0.005971 

8.1548 

Since  this  is  a  two-tailed  test,  the  value  (7=113  would  be 
significant  at  a  level  of  0.01 19 

Rejection  Region 

(a=  0.05) 

Reject  Ho 

As  deseribed  earlier,  the  slaek  veetor  (S)  of  the  S  -  T  heuristie  does  not  provide  an 
effeetive  direetion  into  the  feasible  region  when  the  eonstraint  slaekness  level  is  (2,  2). 
The  dual  direetion  of  S  yields  a  solution  deep  in  the  feasible  region,  eausing  more 
resourees  to  remain.  The  seeond  phase  of  S  -  T  eannot  then  effeetively  return  projeets  to 
the  solution.  However,  TOYODA  and  L  -  M  newly  calculate  the  amount  of  remaining 
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resource  to  select  new  variables  at  every  iteration.  At  the  end,  less  resources  in  the 
constraints  remain  as  compared  to  S  -  T.  The  reason  for  better  solution  of  L  -  M  under 
the  circumstance  of  Si  =  0.7  and  S2  =  0.7  results  from  its  penalty  factor.  The  penalty 
factor  used  is  as  follows: 

Vj  =  Max,^i_ioo{(^4  -ay)l{l-DA.  -  a..)} 

As  we  mentioned  in  Chapter  III,  when  calculating  the  effective  gradient,  L  -  M 
always  picks  the  higher  value  from  the  penalty  cost  between  the  first  and  second 
constraint.  Thus,  choosing  a  higher  value  of  penalty  cost  would  imply  that  L  -  M 
considers  only  the  worst  constraint  from  a  penalty  cost  perspective. 

4.1.5  Implication 

The  form  of  the  effective  gradient,  used  to  either  add  or  drop  variables,  is 
sensitive  to  the  type  of  problem.  In  the  current  setting,  S  -  T  is  better  when  slackness 
levels  differ;  TOYODA  and  L  -  M  are  better  when  both  constraints  are  tight,  and  L  -  M 
is  the  best  when  both  constraints  are  loose.  This  suggests  analyzing  problem 
characteristics  before  selecting  a  heuristic;  a  point  made  by  Loulou  and  Michaelides 
(1979)  and  Hooker  (1994). 

4.2  Correlation 

There  are  45  feasible  correlation  structures  in  the  test  set.  This  section  analyzes 
the  three  heuristics  with  respect  to  correlation  between  objective  function  and  constraint 
coefficients,  as  well  as  correlation  between  constraint  coefficients. 
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4,2.1  Results 


Table  9  presents  counts  of  how  many  times  each  heuristic  yields  the  best  solution 
(excluding  ties)  by  problem  correlation  structure  listed  in  coded  form.  Each  correlation 
structure  contains  20  to  40  samples  (5  replications  over  4  slackness  settings).  Figures  5 
through  9  summarize  the  Table  9  data  graphically  for  various  “slices”  of  the  data;  each 
focuses  on  a  different  level  of  . 


Table  9.  The  number  of  times  best  by  each  heuristic  under  correlation 


Correlation 

Methods 

Toyoda 

S-T 

L-M 

2,2,2 

5 

9 

1 

2,1,1 

7 

6 

5 

2,0,0 

5 

3 

6 

2,-1, -1 

8 

5 

4 

2,-2,-2 

8 

3 

5 

1,2,1 

3 

9 

7 

1,1,2 

1 

4 

0 

1,1,1 

7 

21 

0 

1,1,0 

3 

12 

0 

1,0,1 

8 

8 

2 

1,0,0 

9 

19 

11 

i,o,-i 

3 

10 

7 

i,-i,o 

1 

8 

9 

1,-1,-1 

1 

18 

20 

l,-l,-2 

2 

9 

9 

l,-2,-l 

5 

6 

5 

0,2,0 

3 

4 

9 

0,0,2 

3 

5 

1 

0,1,1 

3 

5 

5 

0,1,0 

3 

19 

17 

o,i,-i 

2 

10 

7 

0,0,1 

9 

16 

1 

0,0,0 

6 

20 

2 

o,-i,o 

2 

12 

20 

o,o,-i 

6 

22 

3 

0,-1, 1 

5 

3 

3 

0,-l,-l 

1 

10 

8 
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Correlation 

Methods 

Toyoda 

S-T 

L-M 

0,0, -2 

1 

11 

2 

0,-2, 0 

1 

3 

10 

-1,2,-1 

12 

3 

3 

-1,1,0 

4 

6 

9 

-1,1,-1 

2 

17 

19 

-l,l,-2 

1 

9 

10 

-1,0,1 

3 

10 

3 

-1,0,0 

1 

21 

17 

-i,o,-i 

1 

8 

8 

-1,-1,2 

1 

0 

3 

-1,-1,1 

6 

10 

9 

-i,-i,o 

2 

8 

4 

-1,-2,1 

0 

2 

7 

-2,2, -2 

9 

4 

6 

-2,1,-1 

6 

5 

4 

-2,0,0 

3 

5 

9 

-2,-l,l 

2 

3 

7 

-2,-2,2 

0 

1 

0 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0 


Ml 


1 


2,2,2  2,1,1  2,0,0  2,-1,-1  2,-2,-2 


□TOYODA 
■  S-T 
□  L-  M 


Figure  5.  Performance  of  the  three  heuristic  methods  when  =2(1) 
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Figure  6.  Performance  of  the  three  heuristic  methods  when  PcA'  =  ^  (2) 


□TOYODA 

■ 

□  L-  M 

_ 
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Figure  7.  Performance  of  the  three  heuristic  methods  when  PcA'  =  °  (3) 
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Figure  8.  Performance  of  the  three  heuristic  methods  when  PCA'  = 


Figure  9.  Performance  of  the  three  heuristic  methods  when  =  -2  (5) 
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4,2.2  Chi  -  square  test 


Table  10  summarizes  the  results  of  a  Chi  -  Square  test  on  each  correlation 


structure,  and  for  those  that  have  it,  a  best  performing  heuristic.  Clearly  S  -  T  does  very 


well  compared  to  TOYODA  or  L  -  M. 


Table  10.  Chi-square  test  for  correlations 


Correlation 

df 

Probability 

Reject  (a  =  0.1) 

Best 

2,2,2 

6.4 

2 

0.040762 

Reject  Ho 

H 

1 

2,1,1 

0.333333 

2 

0.846482 

2,0,0 

1 

2 

0.606531 

2,-l,-l 

1.529412 

2 

0.465471 

2,-2,-2 

2.375 

2 

0.304983 

1,2,1 

2.947368 

2 

0.22908 

1,1,2 

5.2 

2 

0.074274 

Reject  Ho 

S-T 

1,1,1 

24.5 

2 

4.79E-06 

Reject  Ho 

S-T 

1,1,0 

15.6 

2 

0.00041 

Reject  Ho 

S-T 

1,0,1 

4 

2 

0.135335 

1,0,0 

4.307692 

2 

0.116037 

1,0,-1 

3.7 

2 

0.157237 

1,-1,0 

6.333333 

2 

0.042144 

Reject  Ho 

S  -  T,  L  -  M 

1,-1,-1 

16.76923 

2 

0.000228 

Reject  Ho 

S  -  T,  L  -  M 

l,-l,-2 

4.9 

2 

0.086294 

Reject  Ho 

S  -  T,  L  -  M 

l,-2,-l 

0.125 

2 

0.939413 

0,2,0 

3.875 

2 

0.144064 

0,0,2 

2.666667 

2 

0.263597 

0,1,1 

0.615385 

2 

0.735141 

0,1,0 

11.69231 

2 

0.002891 

Reject  Ho 

S  -  T,  L  -  M 

0,1,-1 

5.157895 

2 

0.075854 

Reject  Ho 

S  -  T,  L  -  M 

0,0,1 

13 

2 

0.001503 

Reject  Ho 

S-T 

0,0,0 

19.14286 

2 

6.97E-05 

Reject  Ho 

S-T 

0,-l,0 

14.35294 

2 

0.000764 

Reject  Ho 

L-M 

o,o,-i 

20.19355 

2 

4.12E-05 

Reject  Ho 

S-T 

0,-l,l 

0.727273 

2 

0.695144 

0,-l,-l 

7.052632 

2 

0.029413 

Reject  Ho 

S  -  T,  L  -  M 

0,0, -2 

13 

2 

0.001503 

Reject  Ho 

S-T 

0,-2,0 

9.571429 

2 

0.008348 

Reject  Ho 

L-M 

-1,2,-1 

9 

2 

0.011109 

Reject  Ho 

TOYODA 

-1,1,0 

2 

2 

0.367879 

-1,1,-1 

13.63158 

2 

0.001096 

Reject  Ho 

S  -  T,  L  -  M 

-l,l,-2 

7.3 

2 

0.025991 

Reject  Ho 

S  -  T,  L  -  M 

-1,0,1 

6.125 

2 

0.046771 

Reject  Ho 

S-T 

-1,0,0 

17.23077 

2 

0.000181 

Reject  Ho 

S  -  T,  L  -  M 

-i,o,-i 

5.764706 

2 

0.056003 

Reject  Ho 

S  -  T,  L  -  M 

-1,-1, 2 

3.5 

2 

0.173774 

-1,-1, 1 

1.04 

2 

0.594521 

-i,-i,o 

4 

2 

0.135335 

-1,-2,1 

8.666667 

2 

0.013124 

Reject  Ho 

L-M 

-2,2,-2 

2 

2 

0.367879 

-2,1,-1 

0.4 

2 

0.818731 

-2,0,0 

3.294118 

2 

0.192616 

-2,-1, 1 

3.5 

2 

0.173774 

-1-1,1 

2 

2 

0.367879 

Ho :  The  observed  values  do  not  differ  significantly  from  their  expected  value. 
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We  next  explore  why  a  heuristie  is  better  under  some  eorrelation  structures,  and 
what  makes  heuristics  perform  well  in  general. 

4,2,3  Analysis  of  =  -  0,99997,  =  -  0,99773,  and  p,.  =  0,99752  (-2,  -2,  2) 

C.,A  L-A  A  A 

There  are  specific  combinations  of  negative  correlation  between  objective 
function  coefficients  and  constraints  coefficients,  and  positive  correlation  between 
constraint  coefficients  that  provides  the  best  condition  for  a  heuristic.  In  constructive 
approaches  such  as  the  TOYODA  and  L  -  M  heuristics,  variables  with  the  highest  value 

of  the  effective  gradient  ratio  VL  are  selected,  i.e.,  largest  profit  per  unit  resource 

/  ^  J 

consumed.  For  a  large  effective  gradient,  cj  should  be  relatively  large  and  Vj  should  be 
relatively  small  with  p^^,  =  -  0.99997,  p^^2  =- 0.99773,  both  constraints  consume 

relatively  less  resources  while  profit  is  relatively  large.  Conversely,  when  profit  is  small, 
resource  consumption  will  be  large.  Since  the  value  of  yC>^,^2  is  close  to  one,  and  027 

have  similar  relationships  in  the  problems.  This  makes  the  problems  easy  for  heuristics 
to  find  best  results.  TOYODA  and  L  -  M  found  17  optimal  solutions  and  S  -  T  found  19 
optimal  solutions  out  of  the  20  test  problems  as  shown  below  in  Table  1 1 .  Similarly, 
conditions  of  weaker  negative  correlation  between  objective  function  coefficients  and 
both  constraints’  coefficients,  and  stronger  positive  constraint  coefficient  correlation  (-1, 

Table  11.  Number  of  times  optimum  found  by  each  heuristic  under  correlation  structure 

-2,  -2,  2 


Correlation 

TOYODA 

S-T 

L-M 

-2,  -2,  2 

17 

19 

17 
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-1,2)  also  give  good  conditions  for  each  heuristic.  Under  these  conditions,  the  choice  of 
heuristics  does  not  matter  since  all  do  well. 

4,2,4  Analysis  of  Toyoda’s  heuristic 


Figure  10.  Better  Performance  of  TOYODA  method  under  various  correiations 

As  shown  in  Figure  10  above,  the  TOYODA  heuristic  has  better  performance 

when  p  ,  and  p  ^  have  opposite  sign  of  correlation  and  p  ^  2  has  negative  correlation; 
for  example  p„^,  =  -  1 ,  ,2  =  2,  and  /?  ,  ^2  =  -  1  •  In  other  words,  if  the  profit  is  large, 

the  first  constraint  coefficient  is  relatively  larger  and  the  second  constraint  coefficient  is 
relatively  smaller.  The  TOYODA  heuristic  always  picks  the  highest  effective  gradient. 
As  mentioned  earlier,  to  have  a  higher  and  effective  gradient,  relative  profit  should  be 
large  and  penalty  cost  should  be  small.  When  constraint  slackness  is  tight  and  tight  or 
loose  and  loose  Pu  increases  in  almost  same  value  at  each  iteration,  since  Pu  consist  of 
value  and  of  selected  variables.  Since  the  penalty  cost  is  multiplying 

i^Selected  ie^Selected 

(P. -P) 

by  Pi'.  U I  =  — ,  for  small  penalty  cost,  relatively  small  values  of  an  plus  are 
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picked.  Therefore,  the  correlations  have  opposite  signs  and  one  value  is  larger  than  the 
other  making  the  sum  beeome  a  relatively  small  number.  This  means  choosing  the  least 
resouree  consuming  variable,  meaning  more  variables  ean  be  selected  whieh  in  turn 
results  in  better  objective  funetion  value. 

4,2,5  Analysis  of  Senju-Toyoda’s  heuristic 


Figure  11.  Better  Performance  of  S  -  T  method  under  various  p^\^i  correiations  (1) 


□TOYODA 

■ 

□  L-  M 


Figure  12.  Better  Performance  of  S  -  T  method  under  various  correiations  (2) 
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When  the  and  are  near  zero  or  mid  -  range,  S  -  T  appears  to  dominate. 

However,  for  each  of  the  correlation  level  in  Figure  1 1  and  Figure  12  (exception  is  2,2,2), 
40  test  problems  were  solved.  This  doubling  of  test  problems  over  the  other  34 
correlation  levels  was  an  artifact  of  the  Hill  and  Reilly  (2000)  experimental  design. 

When  independence  or  low  correlation  levels  exist  between  constraint  coefficients  and 
objective  function  coefficients,  the  striking  difference  is  caused  by  the  S  -  T  method’s 
better  performance  on  mixed  constraint  slackness  settings;  half  of  all  problems  are  mixed 
constraints,  so  S  -  T  should  have  better  results  when  there  are  no  dominant  correlations. 
4,2,6  Analysis  of  Loulou  -  Michaelides’  heuristic 


Figure  13.  Better  Performance  of  L  -  M  method  under  various  correlations 

L  -  M  has  a  tendency  to  have  a  better  performance  when  any  one  constraint  has 

strongly  negative  correlation  (p^^i  =-0.99997  or  =- 0.99773)  with  the  objective 

function.  As  previously  described,  the  best  condition  for  a  heuristic  is  strong  negative 
correlation  between  objective  function  coefficients  and  both  constraints’  coefficients  and 
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strong  positive  correlation  between  constraint  coefficients.  L  -  M  heuristic  always  picks 
a  penalty  factor  based  on  the  worst-case  constraint,  i.e.,  it  consumes  more  resources. 

Thus,  inter  -  constraint  correlation  does  not  affect  heuristic  performance  because  L  -  M 
only  considers  one  constraint.  As  shown  above  in  Figure  13,  L  -  M  has  the  best 
performance  on  any  combination  with  strong  negative  correlation  between  objective 
function  coefficients  and  coefficients  of  one  of  the  constraints.  However,  even  though 
one  of  constraints  has  strongly  negative  correlation  with  profits,  TOYODA  has  better 
results  when  two  constraints  have  the  opposite  sign  of  correlation  such  as  negative 
correlation  and  positive  correlation  with  profits. 

4,2,7  Implication 

When  correlation  structure  has  strongly  negative  correlation  between  profits  and 
constraint  coefficients,  primal  effective  methods  perform  better  than  dual  methods. 
Especially,  when  opposite  signs  of  correlation  exist,  TOYODA’ s  performance  is 
distinguishable.  Since  L  -  M  is  not  affected  by  correlation  of  inter-constraints,  L  -  M  has 
better  solutions  when  any  one  constraint  has  strongly  negative  correlation  with  profits. 
Except  in  the  above  condition,  S  -  T  yields  more  solutions  close  to  the  optimum. 
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V.  New  Heuristic  Comparison 


5.1  Introduction 

The  previous  chapter  examined  heuristic  performance  as  a  function  of  problem 
characteristics;  in  particular,  when  is  a  heuristic  the  best  choice  for  a  problem?  In  this 
chapter,  a  New  Combined  Heuristic  examines  problem  structure  and  runs  a  “best” 
heuristic  based  on  the  combination  of  constraint  slackness  and  correlation  structure  used 
in  Chapter  IV.  The  section.  New  Developed  Heuristic,  provides  a  new  heuristic  where 
parameters  have  been  changed  based  on  the  analysis  in  Chapter  IV. 

5.2  New  Combined  Heuristic 

Hooker  (1994)  as  well  as  Loulou  and  Michaelides  (1979)  suggest  basing  heuristic 
choice  on  computed  problem  characteristics.  In  Chapter  IV,  the  best  heuristic  among 
three  different  heuristics,  TOYODA,  S  -  T,  and  L  -  M,  was  determined  based  on 
constraint  slackness  and  correlation  structure.  When  both  constraints  are  tight, 

TOYODA  and  L  -  M,  primal  effective  methods,  are  best.  When  constraint  slackness  is 
mixed,  S  -  T  heuristic  is  best.  Finally,  the  performance  of  L  -  M  is  the  best  when  both 
constraints  are  loose.  Table  12  summarizes  the  results. 

For  correlation  structures,  TOYODA  had  the  better  performance  when  the  sign  of 
correlation  of  objective  function  coefficients  and  constraint  coefficients  is  opposite,  for 
example  (2,  -1,  -1).  L  -  M  was  better  when  any  one  constraint  has  strongly  negative 
correlation  with  objective  function  coefficients.  Table  13  summarizes  the  results. 
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Finally,  a  special  combination  between  constraint  slackness  and  correlation 
structure  dominates  the  specific  best  heuristic.  For  example,  when  correlation  is  (-2,  1, 
-1),  TOYODA  is  the  best  heuristic.  However,  when  this  correlation  is  combined  with 
constraint  slackness  (2,  2),  L  -  M  is  the  best  heuristic.  The  20  combinations  between 
correlation  structure  and  constraint  slackness,  which  dominate  the  specific  method  as  the 
best  heuristic,  were  found  and  are  summarized  in  Table  14. 

Our  typology  was  coded  and  run  against  the  problem  set.  This  new  combined 
heuristic  code  is  available  in  Appendix  D. 

Table  12.  Dominant  Constraint  Slackness  and  the  Best  Heuristic 


Heuristic 

Constraint  Slackness  1 

Constraint  Slackness  2 

L-M 

Tight 

Tight 

S-T 

Tight 

Loose 

S-T 

Loose 

Tight 

L-M 

Loose 

Loose 

Table  13.  Dominant  Correlation  Structures  and  the  Best  Heuristic 


Heuristic 

CAl 

CA2 

A1A2 

TOYODA 

2 

-1 

-1 

TOYODA 

2 

-2 

-2 

TOYODA 

1 

1 

2 

TOYODA 

-1 

2 

-1 

TOYODA 

-2 

2 

-2 

TOYODA 

-2 

1 

-1 

S-T 

0 

0 

2 

S-T 

0 

0 

1 

S-T 

0 

0 

0 

S-T 

0 

0 

-1 

S-T 

0 

0 

-2 

L-M 

0 

-2 

0 

L-M 

-1 

-2 

1 

L-M 

-2 

0 

0 

L-M 

-2 

-1 

1 
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Table  14.  Dominant  Combination  between  Constraint  Slackness  and  Correlation 

Structure,  and  the  Best  Heuristic 


Heuristic 

Slackness  1 

Slackness2 

CAl 

CA2 

A1A2 

TOYODA 

1 

1 

2 

2 

2 

TOYODA 

1 

1 

2 

1 

1 

TOYODA 

1 

1 

1 

1 

1 

TOYODA 

1 

1 

1 

1 

0 

TOYODA 

1 

1 

1 

0 

1 

TOYODA 

1 

1 

1 

0 

0 

TOYODA 

1 

1 

1 

-2 

-1 

TOYODA 

1 

1 

-1 

-1 

2 

TOYODA 

2 

2 

1 

1 

1 

TOYODA 

2 

2 

1 

1 

0 

TOYODA 

2 

2 

0 

0 

1 

TOYODA 

2 

2 

0 

-1 

1 

S-T 

1 

2 

0 

-2 

0 

S-T 

1 

2 

-1 

-2 

1 

S-T 

2 

1 

-2 

0 

0 

S-T 

2 

1 

-2 

-1 

1 

S-T 

2 

2 

-1 

0 

1 

L-M 

1 

2 

-2 

2 

-2 

L-M 

1 

2 

-2 

1 

-1 

L-M 

2 

2 

-2 

1 

-1 

5.2,1  Results  under  Constraint  Slackness 

The  performance  of  the  new  combined  heuristic  is  summarized  in  Table  15  and 
Figure  14.  Table  15  and  Figure  14  show  that  the  new  combined  heuristic  has  slightly 
better  performance  than  the  prior  best  heuristic  under  various  constraint  slackness  levels 
and  has,  overall,  more  consistent  performance.  Since  the  new  combined  heuristic  runs 
the  best  heuristic  under  certain  dominant  constraint  slackness  levels  and  correlation 
structures,  the  new  combined  heuristic  in  Table  15  shows  the  number  counted  when  the 
prior  best  heuristic  and  the  new  combined  heuristic  have  the  same  value. 
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Figure  14.  Performance  of  new  combined  method  under  various  constraint  slackness 
Table  15.  The  number  of  times  best  by  each  heuristic  under  constraint  slackness 


Constraint 

Slackness 

Method 

Toyoda 

S-T 

L-M 

New 

Combined 

1,1 

87 

15 

123 

164 

1,2 

22 

186 

15 

188 

2,1 

24 

184 

13 

182 

2,2 

41 

17 

147 

157 

5,2,2  Results  under  Correlation  Structure 

The  new  approach  does  particularly  well  over  correlation  structure.  When  we 
count  the  number  of  same  solution  values  of  best  heuristic  and  the  new  combined 
heuristic,  the  new  combined  heuristic  performed  better  than  the  previous  best  heuristic. 
Among  45  feasible  correlation  structures,  the  new  combined  heuristic  is  best  in  33 
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correlation  structures.  Figures  15  through  19  plot  the  data  similar  to  Figures  5  through  9. 
The  key  trend  to  note  is  the  consistent  levels  of  performance  by  the  combined  heuristic. 


Figure  15.  Performance  of  new  combined  heuristic  when  =2(1) 


Figure  16.  Performance  of  new  combined  heuristic  when  PcA'  =  (2) 
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Figure  17.  Performance  of  new  combined  heuristic  when  PCA'  =  0  (3) 


□TOYODA 

■ 

□  L-M 
H 


Figure  18.  Performance  of  new  combined  heuristic  when  PcA'  = 


□TOYODA 

■ 

□  L-M 
n 


-2, 2, -2  -2, 1,-1  -2,0,0  -2,-1, 1  -2,-2,2 


Figure  19.  Performance  of  new  combined  heuristic  when  PcA'  =  "2  (5) 
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Table  16  breaks  the  performance  numbers  out  by  each  of  the  45  correlation 
structures  in  the  test  problem  set.  Clearly,  the  combined  heuristic  does  well. 

Table  16.  The  number  of  times  best  by  the  new  combined  heuristics  under  correlation 

structure 


Correlation 


Heuristics 


Tovoda 


5 

7 

5 


L-M 


1 

5 

6 

4 

5 


New  Combined 


9 

7 

16 

4 

0 

1 

21 

0 

24 

12 

0 

13 

8 

2 

14 

19 

11 

32 

10 

7 

17 

8 

9 

17 

18 

20 

37 

9 

9 

18 

6 

5 

14 

4 

9 

12 

5 

1 

5 

5 

5 

10 

19 

17 

36 

10 

7 

17 

16 

1 

20 

20 

2 

20 

12 

20 

30 

22 

3 

22 

3 

3 

5 

10 

8 

16 

11 

2 

11 

3 

10 

13 

3 

3 

12 

6 

9 

13 

17 

19 

36 

9 

10 

19 

10 

3 

12 

21 

17 

35 

8 

8 

15 

0 

3 

2 

10 

9 

17 

8 

4 

10 

2 

7 

9 

4 

6 

12 

5 

4 

9 

5 

9 

13 

3 

7 

10 

5,3  Cho  Heuristic 


As  we  analyzed  the  best  heuristie  in  Chapter  IV,  the  primal  effeetive  gradient 
methods  sueh  as  TOYODA  and  L  -  M  performed  well  when  the  constraint  slackness 
levels  were  equal.  The  dual  effective  gradient  method  was  the  best  when  the  constraint 
slackness  levels  were  mixed.  L  -  M,  an  improved  version  of  the  TOYODA  heuristic, 
always  considers  the  worst  constraint,  so  it  is  not  as  affected  by  inter-constraint 
correlation.  Therefore,  we  combined  characteristics  of  S  -  T,  into  the  L  -  M  heuristic. 

The  Cho  heuristic  is  thus  extended  from  L  -  M  heuristic  based  on  our  knowledge  of  S  -  T 
from  Chapter  IV. 

The  following  algorithm  is  a  general  explanation  of  the  Cho  heuristic  and  a 
comparison  with  Loulou  and  Michaelides’  method.  Our  method  differs  in  defining  the 
penalty  factor  in  step  3.  We  used  the  same  symbols  as  Toyoda  (1975):  Pi  =  project  i,  i  = 

1 ,  ...,n;Rj  =  restricted  resource  j,j  =  1 ,  . . . ,  m;  T=  Set  of  all  projects;  Tu  =  set  of 
projects  accepted  so  far;  Td  =  set  of  projects  not  in  Tu,  T-  Tu;  Tc  =  set  of  candidate 
projects;  Cy=  total  quantity  of  Rj  required  by  the  set  of  accepted  projects, 

.  Pu  =  cumulative  total  quantity  vector  (Toyoda,  1975).  Since  m  =  2, 

Pu  =  {Ci,  C2).  The  vector  Sj  is  the  surplus  amount  which  is  to  subtract  RHS  of  each 
constraint  from  sum  of  all  row  constraint  coefficients  for  the  appropriate  row.  Define  S  = 
slack  vector  as  used  in  S  -  T  heuristic,  i.e.,  S  =  {Si,  S2)  and  RHS  values,  B  =  {\,  1);  Ai  = 
vector  of  constraint  coefficients,  Ai  =  {an,  au). 

Step  P.  Initialization. 

Tu  =  0,  Td=  T,  Pu  =  Zero  Vector 
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Z=0,  Xi  =  0,  /=!,.. .,100 

Step  2\  Assign  all  candidate  projects  to  Tc,  candidate  projeet  set. 

Tc  =  {P.  \P-  and  A.  <  \-Pu) 

lf7c=  0  STOP 

Otherwise  go  to  Step  3. 

Step  3:  Compute  effeetive  gradient  for  projeets  in  Tc  as  follows: 

(a)  If  Pu  is  a  zero  veetor  then: 

G,.  =c,.  •  V2/(a,j  +<2,2)  (Toyoda  1975) 

(b)  Otherwise,  compute; 

G,  =max,^,2{(^.;  ■ayCj)/^Cf+C^,  } 

G.=p/U^ 

Step  4\  k  =  arg  max  {  G,- 1  Pi  e  Tc  and  feasible  } 

Step  5:  Calculate: 

Z  =  Z  +  Ck,  Xk  =  1 ,  Tu  =  Tu  +  {  Pk} 

Pu  =  Pu  +  Pk,  TD=T-{Pk} 

Go  to  Step  2. 

Each  iteration  seleets  a  new  projeet  with  the  largest  gradient.  For  the  penalty  cost, 
Ui,  multiplying  slack  vector,  Sj,  and  constraint  coefficients,  atj,  provides  a  better  direetion 
especially  when  two  constraints  greatly  differ.  The  multiplier  Cj  eonsiders  the 
cumulative  amount  added  so  far  to  the y'th  constraint.  For  example,  if  the  eharaeteristie  of 
a  problem  is  tight  and  loose.  Si  should  be  large  and  S2  should  be  small.  Sinee  a  greedy 
heuristie  always  selects  the  projeet  with  the  largest  gradient,  it  selects  the  projeet  whose 
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objective  function  coefficient  c,  is  relatively  large  while  constraint  coefficients  ay  is 
relatively  small.  In  this  case,  an  is  generally  larger  than  aa  because  the  sum  of  an  is 
much  larger  than  the  sum  of  aa.  So,  we  are  concerned  with  only  an,  not  aa-  Multiplying 
and  Cl  by  an  make  us  consider  only  the  tight  constraint.  This  enables  a  better 
direction  in  the  feasible  region  considering  the  tight  constraint.  The  code  is  available  in 
appendix  E. 

5,3.1  General  Results 

The  overall  result  of  the  new  developed  method  is  better  than  each  of  three 
heuristics  as  shown  below  in  Table  17.  The  reason  is  appealing  since  the  Cho  heuristic  is 
a  general  purpose  greedy  approach  not  tied  to  problem  type  analysis. 


Table  17.  Comparison  of  CHO  heuristic  with  test  heuristics 


Versus  Heuristic 

BETTER 

TIE 

WORSE 

TOYODA 

714 

107 

299 

S-T 

557 

215 

348 

E-M 

774 

70 

276 

5,3,2  Results  under  various  constraint  slackness 

The  Cho  heuristic  really  improves  primal  heuristic  performance  when  constraint 
slackness  levels  are  mixed.  Even  though  this  method  is  based  on  the  E  -  M  heuristic,  this 
method  is  significantly  better  than  TOYODA  or  E  -  M  at  constraint  slackness,  (1,  2),  and 
(2,  1).  As  we  may  expect,  the  slack  vector  enables  the  heuristic  to  use  resources  more 
effectively  at  constraints.  The  results  are  graphed  in  Eigure  20  and  summarized  in  Table 
18.  Eor  comparison,  Cho  heuristic  in  Table  18  shows  the  number  counted  when  the  prior 
best  heuristic  and  the  Cho  heuristic  have  the  same  or  better  value. 
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Figure  20.  Performance  of  Cho  heuristic  under  various  constraint  siackness 


Tabie  18.  The  number  of  times  best  by  each  heuristic  under  constraint  siackness 


Constraint 

Slackness 

Method 

TOYODA 

S-T 

L-M 

CHO 

1,1 

87 

15 

123 

101 

1,2 

22 

186 

15 

128 

2,1 

24 

184 

13 

126 

2,2 

41 

17 

147 

70 
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VI. 


Conclusion 


6.1  Summary 

The  purpose  of  this  researeh  was  to  understand  what  makes  a  heuristie  perform 
better.  To  aeeomplish  this  goal,  1120  problems  were  examined  based  on  problem 
eonstraint  slackness  setting  and  correlation  structure.  The  research  focused  on  which 
heuristic  gave  the  best  solution  under  these  various  conditions.  The  methodology 
included  two  non-parametric  statistic  tests  to  prove  whether  or  not  any  heuristic  method 
was  significantly  better  than  the  other  methods.  In  the  results  and  analyses  chapter,  the 
best  heuristic  was  examined.  We  also  studied  why  the  best  heuristic  behaved  as  it  did  as 
a  function  of  problem  characteristics.  Using  this  knowledge  we  examined  two  new 
heuristics:  one  based  on  problem  type,  a  second  based  on  a  new  penalty  vector.  These 
new  heuristic  were  competitively  tested  against  the  three  original  heuristics  and 
performed  quite  well. 

6.2  Findings 

Three  heuristics,  TOYODA;  S  -  T;  L  -  M,  were  examined  under  different 
combinations  of  constraint  slackness  and  correlation  structure.  For  a  constraint  slackness 
of  tight  and  tight,  the  primal  effective  gradient  methods  such  as  TOYODA  and  L  -  M 
perform  well  because  these  methods  evaluate  their  effective  gradient  at  each  iteration.  It 
is  very  important  to  point  out  that  TOYODA  and  L  -  M  selects  the  next  variable  to  set  to 
1  by  recalculating  the  remaining  resources  which  is  the  reason  for  yielding  better  solution 
s  in  comparison  with  S  -  T.  For  mixed  constraint  slackness,  S  -  T  is  best  because  its 
slack  vector,  S,  provides  a  better  direction  into  the  feasible  region. 
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For  correlation  structure,  Toyoda’s  method  has  better  results  when  and 
have  the  opposite  sign  of  correlation  and  has  negative  correlation.  Since 

TOYODA  always  evaluates  the  penalty  cost  at  each  iteration,  the  resource  is  used  to  keep 
a  balance  between  two  constraints,  even  though  correlation  has  the  opposite  signs.  The  S 
-  T  method  does  not  have  any  dominant  correlation  structure.  However,  when 
correlation  does  not  exist  among  coefficients,  S  -  T  appears  to  yield  better  solution.  This 
is  mainly  caused  by  the  S  -  T  method’s  better  performance  on  mixed  constraint  settings. 

L  -  M  has  the  merit  that  it  considers  one  of  the  constraints,  and  is  not  affected  by  • 

Thus,  when  or  p^^^  have  strongly  negative  correlation,  L  -  M  has  better 
performance. 

A  meta-heuristic  based  on  problem  characteristics  works  well.  Our  approach 
employed  an  analysis  of  constraint  slackness  and  correlation  structure  to  choose  a  “best” 
heuristic.  Test  results  confirmed  overall  consistent  results. 

Knowledge  gained  from  the  empirical  analysis  led  to  an  improved  primal  greedy 
heuristic.  Both  TOYODA  and  L  -  M  struggled  with  mixed  slackness  settings  while  S  - 
T  did  not.  A  new  penalty  factor  that  incorporated  S  -  T  characteristics  evened  out  the 
primal  heuristic  performance. 

6.3  Recommendations 

There  are  several  areas  that  should  be  examined  in  future  research  of  this  topic. 
This  thesis  examined  1120  problems  which  are  2KP  with  100  variables.  One  could 
examine  more  constraints,  more  variables,  and  larger  test  problems.  In  addition,  one 
could  change  the  correlation  induction  method  to  create  more  correlation  settings,  and 
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produce  more  eonstraint  slackness  settings.  Also,  various  types  of  heuristics  could  be 
examined  sueh  as  tabu  seareh,  genetic  algorithms,  simulated  annealing,  or  an  ant  eolony 
optimization  algorithm. 
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Appendix  A,  TOYODA  Heuristic  Code 


Sub  TOYODA 0 

Dim  objcoeff(l  To  100,  1  To  1120) 

Dim  rhscoeff(l  To  2,  1  To  1120) 

Dim  alcoeff(l  To  100,  1  To  1120) 

Dim  a2coeff(l  To  100,  1  To  1120) 

Dim  objvalue(l  To  1120) 

Sheets ( "data" ) .Activate 
Range ("Al") .Activate 
For  k  =  1  To  1120 
For  i  =  1  To  2 

rhscoeff (i,  k)  =  ActiveCell . Of f set ( 0 ,  i  +  3) .Value 
Next  i 

For  i  =  1  To  100 

objcoeff(i,  k)  =  ActiveCell . Of f set ( 1 ,  i  -  1). Value 
alcoeff (i,  k)  =  ActiveCell . Of f set (2 ,  i  -  1)  .Value 

a2coeff (i,  k)  =  ActiveCell . Of f set ( 3 ,  i  -  1) .Value 

Next  i 

objvalue(k)  =  ActiveCell . Of f set ( 0 ,  3) .Value 
ActiveCell . Of f set ( 4 ,  0) .Activate 
Next  k 


Dim  m  As  Integer,  n  As  Integer 

Dim  Pi(l  To  100)  As  Integer,  R1  As  Integer,  R2  As  Integer 
Dim  Ki(l  To  100)  As  Integer 

Dim  Fil(l  To  100)  As  Single,  Fi2(l  To  100)  As  Single 

Dim  Z  As  Integer 

Dim  Xi(l  To  100)  As  Integer 

Dim  Tu(l  To  100)  As  Integer 

Dim  Td(l  To  100)  As  Integer,  PPi(l  To  100,  1  To  2)  As  Single 

Dim  Cl  As  Single,  C2  As  Single 

Dim  PPu(l  To  2)  As  Single 

Dim  B(1  To  2)  As  Integer 

Dim  Tc(l  To  100)  As  Integer 

Dim  Ui(l  To  100)  As  Single,  Gi(l  To  100)  As  Single 


' Step  1 ' 

Sheets ( "Results" ) .Activate 
Range  ("Cl")  .Activate 

For  k  =  1  To  1120 

For  i  =  1  To  100 
Tu(i)  =  0 
Tc(i)  =  0 
Td(i)  =  1 
Xi(i)  =  0 

Fil (i)  =  alcoeff (i,  k)  /  rhscoeff (1,  k) 
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Fi2(i)  =  a2coeff(i,  k)  /  rhscoeff(2,  k) 

Ki(i)  =  objcoeff(i,  k) 

PPi  (i,  1)  =  Fil (i) 

PPi  (i,  2)  =  Fi2 (i) 

Next  i 

PPu(l)  =  0 
PPu(2)  =  0 
R1  =  rhscoeff(l,  k) 

R2  =  rhscoeff(2,  k) 
m  =  0 
Z  =  0 
B(l)  =  1 
B(2)  =  1 

' Step  2 ' 

For  j  =  1  To  100 
For  i  =  1  To  100 
If  Td(i)  =  1  Then 

If  PPi(i,  1)  <=  B(l)  -  PPu(l)  And  PPi(i,  2)  <=  B(2)  -  PPu(2) 
Then 

Tc(i)  =  1 
Else 

Tc(i)  =  0 
End  If 
End  If 
Next  i 


'Step  3' 
n  =  0 

For  i  =  1  To  100 
If  Tc(i)  =  0  Then 
n  =  n  +  1 
End  If 
Next  i 

If  n  =  100  Then 
Exit  For 
End  If 

'  Step  4 ' 

If  PPu(l)  =  0  And  PPu(2)  =  0  Then 
For  i  =  1  To  100 

If  Tc(i)  =  1  Then 

Gi(i)  =  Ki(i)  *  (2  ^  0.5)  /  (Fil(i)  +  Fi2(i)) 
End  If 
Next  i 
Else 

Cl  =  0 
C2  =  0 

For  i  =  1  To  100 

If  Tu(i)  =  1  Then 
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Cl  =  Cl  +  Fil (i) 
C2  =  C2  +  Fi2 (i) 
End  If 
Next  i 


For  i  =  1  To  100 

If  Tc(i)  =  1  Then 

If  Fil(i)  *  Cl  +  Fi2(i)  *  C2  =  0  Then 
Gi(i)  =  100000 
End  If 

Ui(i)  =  (Fil(i)  *  Cl  +  Fi2(i)  *  C2)  /  ((Cl 

0.5) 

Gi  (i)  =  Ki  (i)  /  Ui  (i) 

End  If 
Next  i 
End  If 

'Step  5' 

Max  =  0 

For  i  =  1  To  100 
If  Tc(i)  =  1  Then 

If  Gi (i)  >  Max  Then 

Max  =  Gi  (i) 
m  =  i 
End  If 
End  If 
Next  i 


'Step  6' 

Tu(m)  =  1 

PPu(l)  =  PPu(l)  +  PPi (m,  1) 
PPu(2)  =  PPu(2)  +  PPi (m,  2) 
Z  =  Z  +  Ki  (m) 

Td(m)  =  0 
Tc(m)  =  0 
Xi (m)  =  1 

Range ("cl") .Activate 
Next  j 


Next  k 
End  Sub 


+  C2  ^  2) 
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Appendix  B.  S  -  T  Heuristic  Code 


Sub  senjuO 

Dim  objcoeff(l  To  100,  1  To  1120) 

Dim  rhscoeff(l  To  2,  1  To  1120) 

Dim  alcoeff(l  To  100,  1  To  1120) 

Dim  a2coeff(l  To  100,  1  To  1120) 

Dim  objvalue(l  To  1120) 

Sheets ( "data" ) .Activate 
Range ("Al") .Activate 
For  k  =  1  To  1120 
For  i  =  1  To  2 

rhscoeff (i,  k)  =  ActiveCell . Of f set ( 0 ,  i  +  3) .Value 
Next  i 

For  i  =  1  To  100 

objcoeff(i,  k)  =  ActiveCell . Of f set ( 1 ,  i  -  1). Value 
alcoeff (i,  k)  =  ActiveCell . Of f set (2 ,  i  -  1) .Value 
a2coeff (i,  k)  =  ActiveCell . Of f set ( 3 ,  i  -  1) .Value 
Next  i 

objvalue(k)  =  ActiveCell . Of f set ( 0 ,  3) .Value 
ActiveCell . Of f set ( 4 ,  0) .Activate 
Next  k 


'run  the  senju  heuristic' 

Dim  m  As  Integer,  n  As  Integer,  p  As  Integer 

Dim  Pi(l  To  100)  As  Integer,  R1  As  Integer,  R2  As  Integer 

Dim  Ki(l  To  100)  As  Integer 

Dim  ail(l  To  100)  As  Single,  ai2(l  To  100)  As  Single 

Dim  Z  As  Integer 

Dim  Xi(l  To  100)  As  Integer 

Dim  Sail  As  Single,  Sai2  As  Single 

Dim  SI  As  Single,  S2  As  Single 

Dim  Ni(l  To  100)  As  Single 

Dim  Ui(l  To  100)  As  Single,  Gi(l  To  100)  As  Single 
' Step  1 ' 

Sheets ( "Results" ) .Activate 
Range ("Dl") .Activate 


For  k  =  1  To  1120 

R1  =  rhscoeff (1,  k) 

R2  =  rhscoeff (2,  k) 

Sail  =  0 
Sai2  =  0 

'assign  values  to  the  variables 
For  i  =  1  To  100 
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Xi  (i) 

=  1 

Pi  (i) 

=  1 

ail (i) 

=  alcoef f  ( i , 

k) 

/  rhscoef f ( 1 , 

k) 

*  100 

ai2 (i) 

=  a2coef f ( i , 

k) 

/  rhscoef f (2 , 

k) 

*  100 

Ki  (i) 

=  ob j  coef f ( i , 

k) 

Sail  = 

Sail  +  ail  (i; 

1 

Sai2  = 

Sai2  +  ai2  ( i ; 

1 

Next  i 

51  =  Sail  -  100 

52  =  Sai2  -  100 

'find  Gi' 

For  i  =  1  To  100 

Ni(i)  =  SI  *  ail(i)  +  S2  *  ai2(i) 
Gi ( i )  =  Ki ( i )  /  Ni ( i ) 

Next  i 

For  j  =  1  To  100 

' sort  based  on  Gi 
Min  =  10000 
For  i  =  1  To  100 
If  Pi (i)  =  1  Then 
If  Gi (i)  <  Min  Then 

Min  =  Gi  (i) 
m  =  i 
End  If 
End  If 
Next  i 


'drop  project  with  the  lowest  gradient' 

Pi  (m)  =  0 
Xi (m)  =  0 

51  =  SI  -  ail (m) 

52  =  S2  -  ai2 (m) 

Z  =  0 

If  SI  <=  0  And  S2  <=  0  Then 
For  i  =  1  To  100 

Z=Z+Xi(i)  *Ki(i) 

Next  i 

'place  Gi  in  order 
For  i  =  1  To  100 

If  Xi (i)  =0  Then 

Range ("z2 : zl02") .Cells (i,  1). Value  =  Gi(i) 

Range ( "y2 : yl 02 " ) .Cells(i,  1) .Value  =  i 
End  If 
Next  i 

Range ("Y2 : ZlOl") .Select 

Selection . Sort  Keyl :=Range ("Z2") ,  Orderl : =xlDescending, 
Header : =xlGuess , 
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OrderCustom: =1 ,  MatchCase : =False,  Orientation : =xlTopToBottom 


'add  back  in  a  product 
For  i  =  1  To  100 

p  =  Range ("y2 :Y102") .Cells(i,  1) .Value 
If  p  >  0  Then 

If  ail (p)  <=  -SI  And  ai2 (p)  <=  -S2  Then 
Pi(p)  =  1 
Xi (p)  =  1 

51  =  SI  +  ail (p) 

52  =  S2  +  ai2  (p) 

Z  =  Z  +  Ki  (p) 

End  If 
End  If 

Next  i 
Exit  For 
End  If 

Next  j 


Next  k 
End  Sub 
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Appendix  C,  L  -  M  Heuristic  Code 


Sub  LOULOUO 

Dim  objcoeff(l  To  100,  1  To  1120) 

Dim  rhscoeff(l  To  2,  1  To  1120) 

Dim  alcoeff(l  To  100,  1  To  1120) 

Dim  a2coeff(l  To  100,  1  To  1120) 

Dim  objvalue(l  To  1120) 

Dim  obj valuehill ( 1  To  1120) 

Dim  i  As  Integer,  j  As  Integer,  k  As  Integer 

Sheets ( "data" ) .Activate 
Range ("Al") .Activate 
For  k  =  1  To  1120 
For  i  =  1  To  2 

rhscoeff (i,  k)  =  ActiveCell . Of f set ( 0 ,  i  +  3) .Value 
Next  i 

For  i  =  1  To  100 

objcoeff(i,  k)  =  ActiveCell . Of f set ( 1 ,  i  -  1). Value 
alcoeff (i,  k)  =  ActiveCell . Of f set (2 ,  i  -  1) .Value 
a2coeff (i,  k)  =  ActiveCell . Of f set ( 3 ,  i  -  1) .Value 
Next  i 

objvalue(k)  =  ActiveCell . Of f set ( 0 ,  3) .Value 
objvaluehill (k)  =  ActiveCell . Of f set ( 0 ,  13) .Value 
ActiveCell . Of f set ( 4 ,  0) .Activate 

Next  k 

'run  the  L-  M  heuristic' 

Dim  m  As  Integer,  n  As  Integer 

Dim  Pi(l  To  100)  As  Integer,  R1  As  Integer,  R2  As  Integer 
Dim  Ki(l  To  100)  As  Integer 

Dim  Fil(l  To  100)  As  Single,  Fi2(l  To  100)  As  Single 

Dim  Z  As  Integer 

Dim  Xi(l  To  100)  As  Integer 

Dim  Tu(l  To  100)  As  Integer 

Dim  Td(I  To  100)  As  Integer,  PPi(I  To  100,  1  To  2)  As  Single 

Dim  DAI  As  Single,  DA2  As  Single 

Dim  PPu(l  To  2)  As  Single 

Dim  B(1  To  2)  As  Integer 

Dim  Tc(l  To  100)  As  Integer 

Dim  Vi (I  To  100)  As  Single,  Gi(l  To  100)  As  Single 
Dim  Max  As  Single 


' Step  1 ' 

Sheets ( "Results" ) .Activate 
Range ("el") .Activate 

For  k  =  1  To  1120 
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k) 

k) 


For  i  =  1  To  100 
Tu(i)  =  0 
Tc(i)  =  0 
Td(i)  =  1 
Xi (i)  =  0 

Fil  (i)  =  alcoeff(i,  k)  /  rhscoeff(l, 

Fi2(i)  =  a2coeff(i,  k)  /  rhscoeff(2, 

Ki(i)  =  objcoeff(i,  k) 

PPi  (i,  1)  =  Fil (i) 

PPi  (i,  2)  =  Fi2 (i) 

Next  i 

PPu(l)  =  0 
PPu(2)  =  0 
R1  =  rhscoeff(l,  k) 

R2  =  rhscoeff(2,  k) 

DAI  =  0 
DA2  =  0 
m  =  0 
Z  =  0 
B(l)  =  1 
B(2)  =  1 

' Step  2 ' 

For  j  =  1  To  100 
For  i  =  1  To  100 
If  Td(i)  =  1  Then 

If  PPi(i,  1)  <=  B(l)  -  PPu(l)  And  PPi(i,  2)  <=  B(2)  -  PPu(2) 
Then 

Tc(i)  =  1 
Else 

Tc(i)  =  0 
End  If 
End  If 
Next  i 


'Step  3' 
n  =  0 

For  i  =  1  To  100 
If  Tc(i)  =  0  Then 
n  =  n  +  1 
End  If 
Next  i 

If  n  =  100  Then 
Exit  For 
End  If 

'  Step  4 ' 

If  DAI  =  1  Or  DA2  =  1  Then 
Exit  For 
End  If 


68 


SCI  =  0 
SC2  =  0 

For  i  =  1  To  100 

If  Tc(i)  =  1  Then 

SCI  =  SCI  +  Fil  (i) 

SC2  =  SC2  +  Fi2  (i) 

End  If 
Next  i 

For  i  =  1  To  100 

If  Tc(i)  =  1  Then 

VI  =  (DAI  +  Fil(i))  *  ((SCI  -  Fil(i)))  /  ((1  -  DAI 
Fil (i)  +  0.00000001) ) 

V2  =  (DA2  +  Fi2(i))  *  ((SC2  -  Fi2(i)))  /  ((1  -  DA2 
Fi2 (i)  +  0.00000001) ) 

If  VI  >  V2  Then 
Vi (i)  =  VI 
Else 

Vi (i)  =  V2 
End  If 

Gi(i)  =  Ki(i)  /  (Vi(i)  +  0.0000000001) 

End  If 
Next  i 


'Step  5' 

Max  =  0 

For  i  =  1  To  100 
If  Tc(i)  =  1  Then 

If  Gi (i)  >  Max  Then 

Max  =  Gi  (i) 
m  =  i 
End  If 
End  If 
Next  i 


'Step  6' 

Tu(m)  =  1 

DAI  =  DAI  +  Fil(m) 

DA2  =  DA2  +  Fi2  (m) 

PPu(l)  =  PPu(l)  +  PPi (m,  1) 
PPu(2)  =  PPu(2)  +  PPi (m,  2) 
Z  =  Z  +  Ki  (m) 

Td(m)  =  0 
Tc(m)  =  0 
Xi (m)  =  1 

Range ("el") .Activate 
Next  j 
Next  k 
End  Sub 
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Appendix  D,  New  Combined  Heuristic  Code 


Sub  NewCombinedHeuristic ( ) 

Sheets ( "data" ) .Activate 
Range ("Al") .Activate 
For  t  =  1  To  1120 

For  i  =  1  To  2 

rhscoeff(i,  t)  =  ActiveCell . Of f set ( 0 ,  i  +  3) .Value 
Next  i 

For  i  =  1  To  100 

objcoeff(i,  t)  =  ActiveCell . Of f set ( 1 ,  i  -  1). Value 
alcoeff (i,  t)  =  ActiveCell . Of f set (2 ,  i  -  1) .Value 
a2coeff(i,  t)  =  ActiveCell . Of f set ( 3 ,  i  -  1). Value 
Next  i 

objvalue(t)  =  ActiveCell . Of f set ( 0 ,  3) .Value 
ActiveCell . Of f set ( 4 ,  0) .Activate 
Next  t 

Dim  Sobj (1  To  1120)  As  Long 

Dim  Sail(l  To  1120)  As  Long,  Sai2(l  To  1120)  As  Long 
Dim  Ssqobj(l  To  1120)  As  Long,  Ssqail(l  To  1120)  As  Long 

Dim  Ssqai2(l  To  1120)  As  Long,  Spdcal(l  To  1120)  As  Long 

Dim  Spdca2(l  To  1120)  As  Long,  Spdala2(l  To  1120)  As  Long 

Dim  slacknessl(l  To  1120)  As  Single 
Dim  slackness2(l  To  1120)  As  Single 
Dim  nmCAl (1  To  1120) 

Dim  denCAl(l  To  1120) 

Dim  nmCA2 ( 1  To  1120) 

Dim  denCA2(l  To  1120) 

Dim  nmAlA2(l  To  1120) 

Dim  denAlA2(l  To  1120) 

Dim  corrCAl(l  To  1120)  As  Double 

Dim  corrCA2(l  To  1120)  As  Double 

Dim  corrAlA2(l  To  1120)  As  Double 

For  k  =  1  To  1120 
Sail (k)  =  0 
Sai2 (k)  =  0 
Sobj  (k)  =  0 
Ssqobj  (k)  =  0 
Ssqail (k)  =  0 
Ssqai2 (k)  =  0 
Spdcal (k)  =  0 
Spdca2 (k)  =0 
Spdala2 (k)  =  0 

For  i  =  1  To  100 

Sobj (k)  =  Sobj (k)  +  objcoeff(i,  k) 

Sail (k)  =  Sail (k)  +  alcoeff (i,  k) 

Sai2(k)  =  Sai2(k)  +  a2coeff(i,  k) 

Ssqobj (k)  =  Ssqobj (k)  +  (objcoeff(i,  k) )  ^2 

Ssqail (k)  =  Ssqail (k)  +  (alcoeff (i,  k) )  ^2 
Ssqai2 (k)  =  Ssqai2(k)  +  (a2coeff(i,  k) )  ^2 
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Spdcal (k)  =  Spdcal (k)  +  (objcoeff(i,  k)  *  alcoeff(i,  k) ) 
Spdca2 (k)  =  Spdca2 (k)  +  (objcoeff(i,  k)  *  a2coeff(i,  k)  ) 
Spdala2 (k)  =  Spdala2 (k)  +  (alcoeff(i,  k)  *  a2coeff(i,  k) ) 

Next  i 
Next  k 

For  k  =  1  To  1120 


slacknessl (k)  =  rhscoeff(l,  k)  /  Sail (k) 
slackness2 ( k)  =  rhscoeff(2,  k)  /  Sai2 (k) 
nmCAl (k)  =  100  *  Spdcal (k)  -  Sobj (k)  *  Sail (k) 

denCAl(k)  =  ((100  *  Ssqobj (k)  -  ((Sobj(k))  "  2))  "  0.5)  *  ((100  * 
Ssqail (k)  -  ((Sail(k))  ^  2))  ^  0.5) 

nmCA2(k)  =  100  *  Spdca2(k)  -  Sobj(k)  *  Sai2(k) 

denCA2(k)  =  ((100  *  Ssqobj (k)  -  ((Sobj(k))  "  2))  "  0.5)  *  ((100  * 
Ssqai2 (k)  -  ((Sai2(k))  ^2))  ^  0.5) 

nmAlA2(k)  =  100  *  Spdala2(k)  -  Sail(k)  *  Sai2(k) 

denAlA2(k)  =  ((100  *  Ssqail (k)  -  ((Sail(k))  "  2))  "  0.5)  *  ((100  * 

Ssqai2 (k)  -  ((Sai2(k))  ^2))  ^  0.5) 

corrCAl(k)  =  nmCAl(k)  /  denCAl(k) 
corrCA2(k)  =  nmCA2(k)  /  denCA2(k) 
corrAlA2(k)  =  nmAlA2(k)  /  denAlA2(k) 

Next  k 


For  k  =  1  To  1120 

If  corrCAl(k)  >  0.75  And  corrCA2(k)  <  -0.25  And  corrCA2(k)  >  -0.75  And 
corrAlA2(k)  <  -0.25  And  corrAlA2(k)  >  -0.75  Then 
Call  TOYODA(k) 

Elself  corrCAl(k)  >  0.75  And  corrCA2(k)  <  -0.75  And  corrAlA2(k)  <  -0.75 
Then 

Call  TOYODA(k) 

Elself  corrCAl(k)  <  -0.25  And  corrCAl(k)  >  -0.75  And  corrCA2(k)  >  0.75 
And  corrAlA2(k)  <  -0.25  And  corrAlA2(k)  >  -0.75  Then 
Call  TOYODA(k) 

Elself  corrCAl(k)  <  -0.75  And  corrCA2(k)  >  0.75  And  corrAlA2(k)  <  -0.75 
Then 

If  slacknessl (k)  <  0.45  And  slackness2 (k)  >  0.65  Then 
Call  LOULOU(k) 

Else 

Call  TOYODA(k) 

End  If 

Elself  corrCAl(k)  <  -0.75  And  corrCA2(k)  <  0.75  And  corrCA2(k)  >  0.25 
And  corrAlA2(k)  <  -0.25  And  corrAlA2(k)  >  -0.75  Then 
If  slacknessl (k)  >  0.65  And  slackness2 (k)  >  0.65  Then 
Call  LOULOU(k) 

Elself  slacknessl (k)  <  0.45  And  slackness2 (k)  >  0.65  Then 
Call  LOULOU(k) 

Else 

Call  TOYODA(k) 

End  If 

Elself  corrCAl(k)  <  0.75  And  corrCAl(k)  >  0.25  And  corrCA2(k)  >  0.25 
And  corrCA2(k)  <  0.75  And  corrAlA2(k)  >  0.75  Then 
Call  TOYODA(k) 
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Elself  corrCAl(k;)  >  -0.25  And  corrCAl(k;)  <  0.25  And  corrCA2(k;)  >  -0.25 
And  corrCA2(k;)  <  0.25  And  corrAlA2(k;)  <  -0.75  Then 
Call  senju(k) 

Elself  corrCAl(k)  >  -0.25  And  corrCAl(k)  <  0.25  And  corrCA2(k)  >  -0.25 
And  corrCA2(k)  <  0.25  And  corrAlA2(k)  >  -0.75  And  corrAlA2(k)  <  - 
0.25  Then 
Call  senju(k) 

Elself  corrCAl(k)  >  -0.25  And  corrCAl(k)  <  0.25  And  corrCA2(k)  >  -0.25 
And  corrCA2(k)  <  0.25  And  corrAlA2(k)  >  -0.25  And  corrAlA2(k)  < 
0.25  Then 
Call  senju(k) 

Elself  corrCAl(k)  >  -0.25  And  corrCAl(k)  <  0.25  And  corrCA2(k)  >  -0.25 
And  corrCA2(k)  <  0.25  And  corrAlA2(k)  >  0.25  And  corrAlA2(k)  < 
0.75  Then 

If  slacknessl (k)  >  0.65  And  slackness2 (k)  >  0.65  Then 
Call  TOYODA(k) 

Else 

Call  senju(k) 

End  If 

Elself  corrCAl(k)  >  -0.25  And  corrCAl(k)  <  0.25  And  corrCA2(k)  >  -0.25 
And  corrCA2(k)  <  0.25  And  corrAlA2(k)  >  0.75  Then 
Call  senju(k) 

Elself  corrCAl(k)  >  -0.25  And  corrCAl(k)  <  0.25  And  corrCA2(k)  <  -0.75 
And  corrAlA2(k)  >  -0.25  And  corrAlA2(k)  <  0.25  Then 
If  slacknessl (k)  <  0.45  And  slackness2 (k)  >  0.65  Then 
Call  senju(k) 

Else 

Call  LOULOU(k) 

End  If 

Elself  corrCAl(k)  <  -0.75  And  corrCA2(k)  >  -0.25  And  corrCA2(k)  <  0.25 
And  corrAlA2(k)  >  -0.25  And  corrAlA2(k)  <  0.25  Then 
If  slacknessl (k)  >  0.65  And  slackness2 (k)  <  0.45  Then 
Call  senju(k) 

Else 

Call  LOULOU(k) 

End  If 

Elself  corrCAl(k)  <  -0.25  And  corrCAl(k)  >  -0.75  And  corrCA2(k)  <  -0.75 
And  corrAlA2(k)  >  0.25  And  corrAlA2(k)  <  0.75  Then 
If  slacknessl (k)  <  0.45  And  slackness2 (k)  >  0.65  Then 
Call  senju(k) 

Else 

Call  LOULOU(k) 

End  If 

Elself  corrCAl(k)  <  -0.75  And  corrCA2(k)  <  -0.25  And  corrCA2(k)  >  -0.75 
And  corrAlA2(k)  >  0.25  And  corrAlA2(k)  <  0.75  Then 
If  slacknessl (k)  >  0.65  And  slackness2 (k)  <  0.45  Then 
Call  senju(k) 

Else 

Call  LOULOU(k) 

End  If 

Elself  corrCAl(k)  >  0.75  And  corrCA2(k)  >  0.25  And  corrCA2(k)  <  0.75 
And  corrAlA2(k)  >  0.25  And  corrAlA2(k)  <  0.75  Then 
If  slacknessl (k)  <  0.45  And  slackness2 (k)  <  0.45  Then 
Call  TOYODA(k) 

Elself  slacknessl (k)  >  0.65  And  slackness2 (k)  >  0.65  Then 
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Call  LOULOU(k) 


Else 

Call  senju(k) 

End  If 

Elself  corrCAl(k)  >  0.25  And  corrCAl(k)  <  0.75  And  corrCA2(k)  >  0.25 
And  corrCA2(k)  <  0.75  And  corrAlA2(k)  >  0.25  And  corrAlA2(k)  < 
0.75  Then 

If  slacknessl (k)  <  0.45  And  slackness2 (k)  <  0.45  Then 
Call  TOYODA(k) 

Elself  slacknessl (k)  >  0.65  And  slackness2 (k)  >  0.65  Then 
Call  TOYODA(k) 

Else 

Call  senju(k) 

End  If 

Elself  corrCAl(k)  >  0.25  And  corrCAl(k)  <  0.75  And  corrCA2(k)  >  0.25 
And  corrCA2(k)  <  0.75  And  corrAlA2(k)  >  -0.25  And  corrAlA2(k)  < 
0.25  Then 

If  slacknessl (k)  <  0.45  And  slackness2 (k)  <  0.45  Then 
Call  TOYODA(k) 

Elself  slacknessl (k)  >  0.65  And  slackness2 (k)  >  0.65  Then 
Call  TOYODA(k) 

Else 

Call  senju(k) 

End  If 

Elself  corrCAl(k)  >  0.25  And  corrCAl(k)  <  0.75  And  corrCA2(k)  <  0.25 
And  corrCA2(k)  >  -0.25  And  corrAlA2(k)  >  0.25  And  corrAlA2(k)  < 
0.75  Then 

If  slacknessl (k)  <  0.45  And  slackness2 (k)  <  0.45  Then 
Call  TOYODA(k) 

Elself  slacknessl (k)  >  0.65  And  slackness2 (k)  >  0.65  Then 
Call  LOULOU(k) 

Else 

Call  senju(k) 

End  If 

Elself  corrCAl(k)  >  0.25  And  corrCAl(k)  <  0.75  And  corrCA2(k)  <  0.25 

And  corrCA2(k)  >  -0.25  And  corrAlA2(k)  >  -0.25  And  corrAlA2(k)  < 
0.25  Then 

If  slacknessl (k)  <  0.45  And  slackness2 (k)  <  0.45  Then 
Call  TOYODA(k) 

Elself  slacknessl (k)  >  0.65  And  slackness2 (k)  >  0.65  Then 
Call  LOULOU(k) 

Else 

Call  senju(k) 

End  If 

Elself  corrCAl(k)  >  0.25  And  corrCAl(k)  <  0.75  And  corrCA2(k)  <  -0.75 
And  corrAlA2(k)  <  -0.25  And  corrAlA2(k)  >  -0.75  Then 
If  slacknessl (k)  <  0.45  And  slackness2 (k)  <  0.45  Then 
Call  TOYODA(k) 

Elself  slacknessl (k)  >  0.65  And  slackness2 (k)  >  0.65  Then 
Call  LOULOU(k) 

Else 

Call  senju(k) 

End  If 
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Elself  corrCAl(k;)  >  -0.25  And  corrCAl(k;)  <  0.25  And  corrCA2(k;)  <  -0.25 
And  corrCA2(k;)  >  -0.75  And  corrAlA2(k;)  >  0.25  And  corrAlA2(k;)  < 
0.75  Then 

If  slacknessl (k)  <  0.45  And  slackness2 (k)  <  0.45  Then 
Call  LOULOU(k) 

Elself  slacknessl (k)  >  0.65  And  slackness2 (k)  >  0.65  Then 
Call  TOYODA(k) 

Else 

Call  senju(k) 

End  If 

Elself  corrCAl(k)  <  -0.25  And  corrCAl(k)  >  -0.75  And  corrCA2(k)  <  0.25 
And  corrCA2(k)  >  -0.25  And  corrAlA2(k)  >  0.25  And  corrAlA2(k)  < 
0.75  Then 

If  slacknessl (k)  <  0.45  And  slackness2 (k)  <  0.45  Then 
Call  LOULOU(k) 

Else 

Call  senju(k) 

End  If 

Elself  corrCAl(k)  <  -0.25  And  corrCAl(k)  >  -0.75  And  corrCA2(k)  <  -0.25 
And  corrCA2(k)  >  -0.75  And  corrAlA2(k)  >  0.75  Then 
If  slacknessl (k)  <  0.45  And  slackness2 (k)  <  0.45  Then 
Call  TOYODA(k) 

Elself  slacknessl (k)  >  0.65  And  slackness2 (k)  >  0.65  Then 
Call  LOULOU(k) 

Else 

Call  senju(k) 

End  If 

Elself  corrCAl(k)  >  0.75  And  corrCA2(k)  >  0.75  And  corrAlA2(k)  >  0.75 
Then 

If  slacknessl (k)  <  0.45  And  slackness2 (k)  <  0.45  Then 
Call  TOYODA(k) 

Elself  slacknessl (k)  >  0.65  And  slackness2 (k)  >  0.65  Then 
Call  LOULOU(k) 

Else 

Call  senju(k) 

End  If 


Elself  slacknessl (k) 
Call  LOULOU(k) 

<  0.45 

And 

slackness2 

(k) 

<  0.45 

Then 

Elself  slacknessl (k) 
Call  LOULOU(k) 

>  0.65 

And 

slackness2 

(k) 

>  0.65 

Then 

Else 

Call  senju(k) 


End  If 
Next  k 
End  Sub 
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Appendix  E,  Cho  Heuristic  Code 


Sub  ChoHeuristic  ( ) 

Dim  objcoeff(l  To  100,  1  To  1120) 

Dim  rhscoeff(l  To  2,  1  To  1120) 

Dim  alcoeff(l  To  100,  1  To  1120) 

Dim  a2coeff(l  To  100,  1  To  1120) 

Dim  objvalue(l  To  1120) 

Sheets ( "data" ) .Activate 
Range ("Al") .Activate 
For  k  =  1  To  1120 
For  i  =  1  To  2 

rhscoeff (i,  k)  =  ActiveCell . Of f set ( 0 ,  i  +  3) .Value 
Next  i 

For  i  =  1  To  100 

objcoeff(i,  k)  =  ActiveCell . Of f set ( 1 ,  i  -  1). Value 
alcoeff (i,  k)  =  ActiveCell . Of f set (2 ,  i  -  1)  .Value 

a2coeff (i,  k)  =  ActiveCell . Of f set ( 3 ,  i  -  1) .Value 

Next  i 

objvalue(k)  =  ActiveCell . Of f set ( 0 ,  3)  .Value 
ActiveCell . Of f set ( 4 ,  0) .Activate 
Next  k 


'run  Cho  heuristic' 

Dim  m  As  Integer,  n  As  Integer 

Dim  Pi(l  To  100)  As  Integer,  R1  As  Integer,  R2  As  Integer 
Dim  Ki(l  To  100)  As  Integer 

Dim  Fil(l  To  100)  As  Single,  Fi2(l  To  100)  As  Single 

Dim  Z  As  Integer 

Dim  Xi(l  To  100)  As  Integer 

Dim  Tu(l  To  100)  As  Integer 

Dim  Td(l  To  100)  As  Integer,  PPi(l  To  100,  1  To  2)  As  Single 

Dim  Cl  As  Single,  C2  As  Single 

Dim  PPu(l  To  2)  As  Single 

Dim  B(1  To  2)  As  Integer 

Dim  Tc(l  To  100)  As  Integer 

Dim  Ui(l  To  100)  As  Single,  Gi(l  To  100)  As  Single 
Dim  Ul(l  To  100)  As  Single,  U2(l  To  100)  As  Single 


' Step  1 ' 

Sheets ( "Results" ) .Activate 
Range  ("II")  .Activate 

For  k  =  1  To  1120 

For  i  =  1  To  100 
Tu(i)  =  0 
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Tc(i)  =  0 
Td(i)  =  1 
Xi (i)  =  0 

Fil  (i)  =  alcoeff(i,  k)  /  rhscoeff(l,  k) 

Fi2(i)  =  a2coeff(i,  k)  /  rhscoeff(2,  k) 

Ki(i)  =  objcoeff(i,  k) 

PPi  (i,  1)  =  Fil (i) 

PPi  (i,  2)  =  Fi2 (i) 

Next  i 

PPu(l)  =  0 
PPu(2)  =  0 
R1  =  rhscoeff(l,  k) 

R2  =  rhscoeff(2,  k) 
m  =  0 
Z  =  0 
B(l)  =  1 
B(2)  =  1 

' Step  2 ' 

For  j  =  1  To  100 
For  i  =  1  To  100 
If  Td(i)  =  1  Then 

If  PPi(i,  1)  <=  B(l)  -  PPu(l)  And  PPi(i,  2)  <=  B(2)  -  PPu(2) 
Then 

Tc(i)  =  1 
Else 

Tc(i)  =  0 
End  If 
End  If 
Next  i 


'Step  3' 
n  =  0 

For  i  =  1  To  100 
If  Tc(i)  =  0  Then 
n  =  n  +  1 
End  If 
Next  i 

If  n  =  100  Then 
Exit  For 
End  If 


Sail  =  0 

Sai2  =  0 

For  i  =  1  To  100 

Sail  =  Sail  +  Fil (i) 
Sai2  =  Sai2  +  Fi2  (i) 

Next  i 
' Step  4 ' 

51  =  Sail  -  1 

52  =  Sai2  -  1 
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If  PPu(l)  =  0  And  PPu(2)  =  0  Then 
For  i  =  1  To  100 

If  Tc(i)  =  1  Then 

Gi(i)  =  Ki(i)  *  (2  ^  0.5)  /  (Fil(i)  +  Fi2(i)) 

End  If 
Next  i 
Else 

Cl  =  0 
C2  =  0 

For  i  =  1  To  100 

If  Tu(i)  =  1  Then 

Cl  =  Cl  +  Fil  (i) 

C2  =  C2  +  Fi2 (i) 

End  If 

Next  i 

For  i  =  1  To  100 

If  Tc(i)  =  1  Then 

If  Fil(i)  *  Cl  +  Fi2(i)  *  C2  =  0  Then 
Ci(i)  =  100000 
End  If 

Ul(i)  =  (SI  *  Fil(i)  *  Cl)  /  ((Cl  "  2  +  C2  "  2)  "  0.5) 
U2(i)  =  (S2  *  Fi2(i)  *  C2)  /  ((Cl  ^  2  +  C2  ^  2)  ^  0.5) 

If  U1 (i)  >  U2 (i)  Then 

Ui(i)  =  Ul(i) 

Else 

Ui  (i)  =  U2  (i) 

End  If 

Gi  (i)  =  Ki  (i)  /  Ui  (i) 

End  If 
Next  i 
End  If 

'Step  5' 


Max  =  0 

For  i  =  1  To  100 
If  Tc(i)  =  1  Then 

If  Gi (i)  >  Max  Then 

Max  =  Gi  (i) 
m  =  i 
End  If 
End  If 
Next  i 


'Step  6' 

Tu(m)  =  1 

PPu(l)  =  PPu(l)  +  PPi (m,  1) 
PPu(2)  =  PPu(2)  +  PPi (m,  2) 
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Z  =  Z  +  Ki  (m) 

Td(m)  =  0 
Tc(m)  =  0 
Xi (m)  =  1 

Range ("II") .Activate 
Next  j 
Next  k 
End  Sub 
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